Pancreatic cancer segmentation in unregistered multi-parametric MRI with adversarial learning and multi-scale supervision

Automated pancreatic cancer segmentation is crucial for successful clinical aid diagnosis and surgical planning. However, the tiny size and inconspicuous boundaries of pancreatic cancer lesions lead to poor segmentation performance with single-modality imaging. The commonly used registration-based multi-modal fusion strategies may not only introduce uncertainties arising from registration, but also underutilize the complementary information between different modalities. Thus, to achieve different modality-based tumor segmentation, we propose for the first time a registration-free multi-modal and multi-scale adversarial segmentation network (MMSA-Net), which consist of a shared encoder and a dual decoder. Specifically, MMSA-Net combine two complementary modules, inter-modality adversarial learning and intra-modality multi-scale adversarial supervision, to obtain mode-specific segmentation results while facilitating multi-modal fusion. The inter-modality adversarial learning module facilitates the fusion of modality-shared features among different modalities by strengthening the similarity of features extracted by the shared encoder. The intra-modality multi-scale adversarial supervision emphasizes modality-specific features in different decoding paths, inherits and fuses modality-shared features while preserving the feature specificity of each modality, thus outputting competitive modality-specific segmentation results for each modality. Quantitative and qualitative experimental results on multi-parametric MRI pancreatic cancer data show that our method can effectively improve the performance of multi-modal segmentation. The method proposed in this work is expected to be another potential paradigm for addressing multi-modal segmentation tasks in addition to registration. Our source codes will be released at https://github.com/SJTUBME-QianLab/PancreaticCancer_Seg_AdvLearning, once the manuscript is accepted for publication.