Survival Analysis based on Lung Tumor Segmentation using Global Context-aware Transformer in Multimodality.

Lung cancer is the most common type of cancer and is on the rise. Recently, most young people were diagnosed with the disease, accounting for approximately 12% of all cancer patients worldwide. Early diagnosis combined with immediate treatment can increase the chance of successful recovery. We observed that tumor information is essential for diagnosis. However, it is expensive and time-consuming to provide tumor location in practice. We therefore first proposed a tumor segmentation model, Multi-scale Aggregation-based Parallel Transformer Network (MAPTransNet), to segment where the tumor cells are and where the normal tissues are in 3D PET/CT images. In this segmentation model, we employed parallel transformer mechanism to capture global context of multi-scale encoder feature maps and then concatenated them to obtain global context at multi-scale maps. Here, we integrate external attention into original vision transformer (ViT) mechanism to learn position properties of each small patches in the entire dataset. After that, we use output of MAPTransNet model as the region of interest (RoI) image for survival analysis task. Since there is a statistically significant relationship between intensity and size of tumors and disease stage that reflects to individual's survival time, we proposed Multimodality-based Survival Network (MSNet) for predicting hazard rate of non- small cell lung cancer (NSCLC) patients using PET/CT scans with tumor information (RoI image) and clinical data (disease stages, age, histology, etc). The experimental results prove that our proposal achieves competitive performance compared to competing methods in terms of Dice score metric for tumor segmentation task. For survival analysis task, we emphasize that our proposed method outperforms other competing methods in terms of C-index metric. From the results, we conclude that the use of multimodality (clinical and image features) provides rich information related to survival analysis task in NSCLC.