Label-efficient Transformer-Based Framework with Self-Supervised Strategies for Heterogeneous Lung Tumor Segmentation
EXPERT SYSTEMS WITH APPLICATIONS(2025)
Guilin Univ Elect Technol
Abstract
Precise and automatic segmentation of lung tumors is crucial for computer-aided diagnosis and subsequent treatment planning. However, the heterogeneity of lung tumors, varying in size, shape, and location, combined with the low contrast between tumors and adjacent tissues, significantly complicates accurate segmentation. Furthermore, most supervised segmentation models face limitations due to the scarcity and lack of diversity in labeled training data. Although various self-supervised learning strategies have been developed for model pre-training with unlabeled data, their relative benefits for the downstream task of lung tumor segmentation on CT scans remain uncertain. To address these challenges, we introduce a robust and label-efficient Transformer-based framework with different self-supervised strategies for lung tumor segmentation. Our model training is conducted in two phase, during the pre-training phase, we pre-train the model on a large amount of unlabeled CT scans, employing three different pre-training strategies and comparing their impacts on downstream lung tumor segmentation task. In the fine-tuning phase, we utilize the encoders of the pre-trained models for label-efficient supervised fine-tuning. In addition, we design a surrounding samples-based contrastive learning (SSCL) module at the end of the encoder to enhance feature extraction, especially for tumors with indistinct boundaries. Our proposed methods are evaluated on test sets from seven different center. When only a small amount of labeled data is available, compared to supervised models, Ours (SimMIM3D) demonstrates superior segmentation performance on three internal test sets, achieving Dice coefficients of 0.8419, 0.8346, and 0.8282, respectively. Additionally, it also shows strong generalization on external test sets, with Dice coefficients of 0.7594, 0.7684, 0.6578, and 0.6621, respectively. Extensive experiments confirm the efficacy of our methodology, demonstrating significant improvements over recent state-of-the-art supervised segmentation methods in scenarios with limited labeled data. The source code is available at https://github.com/GDPHMediaLab/SSL-Seg.
MoreTranslated text
Key words
Lung tumor segmentation,Vision transformer,Contrastive learning,Masked image modeling,Self-supervised learning
求助PDF
上传PDF
View via Publisher
AI Read Science
AI Summary
AI Summary is the key point extracted automatically understanding the full text of the paper, including the background, methods, results, conclusions, icons and other key content, so that you can get the outline of the paper at a glance.
Example
Background
Key content
Introduction
Methods
Results
Related work
Fund
Key content
- Pretraining has recently greatly promoted the development of natural language processing (NLP)
- We show that M6 outperforms the baselines in multimodal downstream tasks, and the large M6 with 10 parameters can reach a better performance
- We propose a method called M6 that is able to process information of multiple modalities and perform both single-modal and cross-modal understanding and generation
- The model is scaled to large model with 10 billion parameters with sophisticated deployment, and the 10 -parameter M6-large is the largest pretrained model in Chinese
- Experimental results show that our proposed M6 outperforms the baseline in a number of downstream tasks concerning both single modality and multiple modalities We will continue the pretraining of extremely large models by increasing data to explore the limit of its performance
Upload PDF to Generate Summary
Must-Reading Tree
Example

Generate MRT to find the research sequence of this paper
Data Disclaimer
The page data are from open Internet sources, cooperative publishers and automatic analysis results through AI technology. We do not make any commitments and guarantees for the validity, accuracy, correctness, reliability, completeness and timeliness of the page data. If you have any questions, please contact us by email: report@aminer.cn
Chat Paper
Summary is being generated by the instructions you defined