Skip to main content

Research Repository

Advanced Search

LMISA: A Lightweight Multi-modality Image Segmentation Network via Domain Adaptation using Gradient Magnitude and Shape Constraint

Jafari, Mina; Francis, Susan; Garibaldi, Jonathan M.; Chen, Xin

LMISA: A Lightweight Multi-modality Image Segmentation Network via Domain Adaptation using Gradient Magnitude and Shape Constraint Thumbnail


Mina Jafari

Associate Professor


In medical image segmentation, supervised machine learning models trained using one image modality (e.g. computed tomography (CT)) are often prone to failure when applied to another image modality (e.g. magnetic resonance imaging (MRI)) even for the same organ. This is due to the significant intensity variations of different image modalities. In this paper, we propose a novel end-to-end deep neural network to achieve multi-modality image segmentation, where image labels of only one modality (source domain) are available for model training and the image labels for the other modality (target domain) are not available. In our method, a multi-resolution locally normalized gradient magnitude approach is firstly applied to images of both domains for minimizing the intensity discrepancy. Subsequently, a dual task encoder-decoder network including image segmentation and reconstruction is utilized to effectively adapt a segmentation network to the unlabeled target domain. Additionally, a shape constraint is imposed by leveraging adversarial learning. Finally, images from the target domain are segmented, as the network learns a consistent latent feature representation with shape awareness from both domains. We implement both 2D and 3D versions of our method, in which we evaluate CT and MRI images for kidney and cardiac tissue segmentation. For kidney, a public CT dataset (KiTS19, MICCAI 2019) and a local MRI dataset were utilized. The cardiac dataset was from the Multi-Modality Whole Heart Segmentation (MMWHS) challenge 2017. Experimental results reveal that our proposed method achieves significantly higher performance with a much lower model complexity in comparison with other state-of-the-art methods. More importantly, our method is also capable of producing superior segmentation results than other methods for images of an unseen target domain without model retraining. The code is available at GitHub ( to encourage method comparison and further research.

Journal Article Type Article
Acceptance Date Jul 11, 2022
Online Publication Date Jul 13, 2022
Publication Date 2022-10
Deposit Date Aug 1, 2022
Publicly Available Date Aug 2, 2022
Journal Medical Image Analysis
Print ISSN 1361-8415
Electronic ISSN 1361-8423
Publisher Elsevier BV
Peer Reviewed Peer Reviewed
Volume 81
Article Number 102536
Keywords Computer Graphics and Computer-Aided Design; Health Informatics; Computer Vision and Pattern Recognition; Radiology, Nuclear Medicine and imaging; Radiological and Ultrasound Technology
Public URL
Publisher URL


You might also like

Downloadable Citations