Meta Segmentation Network for Ultra-Resolution Medical Images

    Tong Wu
    Tong Wu
    Bicheng Dai
    Bicheng Dai
    Shuxin Chen
    Shuxin Chen

    IJCAI, pp. 544-550, 2020.

    Cited by: 0|Bibtex|Views28|Links
    EI
    Keywords:
    ultra-resolution imagefully connected layersimage segmentationexperimental resultcomputational burdenMore(17+)
    Wei bo:
    We propose Meta Segmentation Network for the effective segmentation of medical ultra-resolution image

    Abstract:

    Despite recent great progress on semantic segmentation, there still exist huge challenges in medical ultra-resolution image segmentation. The methods based on multi-branch structure can make a good balance between computational burdens and segmentation accuracy. However, the fusion structure in these methods require to be designed elabora...More

    Code:

    Data:

    0
    Introduction
    • With the rising up of deep learning, semantic segmentation achieves prominent progress.
    • There are two common ways to process URIs: image downsampling and sliding patches [Altunbay et al, 2009; Chang et al, 2015]
    • The former resizes a large image to a suitable size, e.g., 512 × 512, feeds it into the model, which leads to the great loss of local details, especially for WSIs. The former resizes a large image to a suitable size, e.g., 512 × 512, feeds it into the model, which leads to the great loss of local details, especially for WSIs
    • The latter crops original image into many small patches, segments on patch-level, and combines the segmentation results of these patches.
    • While these methods can effectively reduce the computational burden, the global information provided by spatial context and neighborhood dependency is almost abandoned, which makes it difficult to obtain accurate segmentation results
    Highlights
    • With the rising up of deep learning, semantic segmentation achieves prominent progress
    • We propose a novel multi-branch framework guided by a meta-learning way for ultra-resolution medical image segmentation, namely Meta Segmentation Network (MSN)
    • Memory Feature Pool stores the meta-features of the meta-branch in Mainbody, while MemRM is deployed in non-meta-branch to complement the distinctive features from the non-meta branch with the metafeatures stored in Memory Feature Pool, named Memory
    • We propose Meta Segmentation Network for the effective segmentation of medical ultra-resolution image
    • A novel meta-fusion module with a very simple but effective structure is introduced for branches fusion through a meta-learning way
    • The experimental results on BACH and ISIC demonstrate that our method achieves the best comprehensive performance
    Methods
    • The authors introduce the framework of MSN, which is illustrated in Fig.2.
    • It mainly contains two components: the multi-branch structure (Mainbody), and the Meta Fusion Module (Meta-FM).
    • Mainbody is an all-in-one structure which realizes multi-resolution segments.
    • Unlike the general multi-resolution structure which requires the multiple branches with a special resolution per branch, Mainbody only uses one branch to realize the multi-resolution segmentation.
    Results
    • The authors' method achieves a significant performance improvement over the latest AWMF-CNN, and the overall parameters are close to that of a single branch.
    Conclusion
    • The authors propose MSN for the effective segmentation of medical URIs. A novel meta-fusion module with a very simple but effective structure is introduced for branches fusion through a meta-learning way.
    • MSN achieves a lightweight multi-branch structure with the help of the particular weight sharing mechanism.
    • The experimental results on BACH and ISIC demonstrate that the method achieves the best comprehensive performance
    Summary
    • Introduction:

      With the rising up of deep learning, semantic segmentation achieves prominent progress.
    • There are two common ways to process URIs: image downsampling and sliding patches [Altunbay et al, 2009; Chang et al, 2015]
    • The former resizes a large image to a suitable size, e.g., 512 × 512, feeds it into the model, which leads to the great loss of local details, especially for WSIs. The former resizes a large image to a suitable size, e.g., 512 × 512, feeds it into the model, which leads to the great loss of local details, especially for WSIs
    • The latter crops original image into many small patches, segments on patch-level, and combines the segmentation results of these patches.
    • While these methods can effectively reduce the computational burden, the global information provided by spatial context and neighborhood dependency is almost abandoned, which makes it difficult to obtain accurate segmentation results
    • Methods:

      The authors introduce the framework of MSN, which is illustrated in Fig.2.
    • It mainly contains two components: the multi-branch structure (Mainbody), and the Meta Fusion Module (Meta-FM).
    • Mainbody is an all-in-one structure which realizes multi-resolution segments.
    • Unlike the general multi-resolution structure which requires the multiple branches with a special resolution per branch, Mainbody only uses one branch to realize the multi-resolution segmentation.
    • Results:

      The authors' method achieves a significant performance improvement over the latest AWMF-CNN, and the overall parameters are close to that of a single branch.
    • Conclusion:

      The authors propose MSN for the effective segmentation of medical URIs. A novel meta-fusion module with a very simple but effective structure is introduced for branches fusion through a meta-learning way.
    • MSN achieves a lightweight multi-branch structure with the help of the particular weight sharing mechanism.
    • The experimental results on BACH and ISIC demonstrate that the method achieves the best comprehensive performance
    Tables
    • Table1: Comparison results on BACH. X1 and X2 are the nonmeta-branches in our method, and X3 is the trained meta-branch
    • Table2: Comparison results on ISIC
    • Table3: The effectiveness of the weight sharing of MSN on BACH
    • Table4: The impact of gap layers on our weight sharing. ‘on nongap layers’ denotes that we add Mem-FP and Mem-RM only to the non-gap layers
    • Table5: The effectiveness of meta-fusion on BACH and ISIC. # F denotes the parameter amount of the fusion part of each method
    Download tables as Excel
    Funding
    • This work is supported by the National Natural Science Foundation of China under Grant 61876161, Grant 61772524, Grant U1065252 and partly by the Beijing Municipal Natural Science Foundation under Grant 4182067, and partly by the Fundamental Research Funds for the Central Universities associated with Shanghai Key Laboratory of Trustworthy Computing
    Reference
    • [Altunbay et al., 2009] Dogan Altunbay, Celal Cigir, Cenk Sokmensuer, and Cigdem Gunduz-Demir. Color graphs for automated cancer diagnosis and grading. IEEE Transactions on Biomedical Engineering, 57(3):665–674, 2009.
      Google ScholarLocate open access versionFindings
    • [Aresta et al., 2019] Guilherme Aresta, Teresa Araujo, Scotty Kwok, Sai Saketh Chennamsetty, Mohammed Safwan, Varghese Alex, Bahram Marami, Marcel Prastawa, Monica Chan, Michael Donovan, et al. Bach: Grand challenge on breast cancer histology images. Medical image analysis, 2019.
      Google ScholarFindings
    • [Bottou, 2012] Leon Bottou. Stochastic gradient descent tricks. In Neural networks: Tricks of the trade, pages 421– 436.
      Google ScholarLocate open access versionFindings
    • [Chang et al., 2015] Hang Chang, Yin Zhou, Alexander Borowsky, Kenneth Barner, Paul Spellman, and Bahram Parvin. Stacked predictive sparse decomposition for classification of histology sections. International journal of computer vision, 113(1):3–18, 2015.
      Google ScholarLocate open access versionFindings
    • [Chen et al., 2014] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv preprint arXiv:1412.7062, 2014.
      Findings
    • [Chen et al., 2018] Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pages 801– 818, 2018.
      Google ScholarLocate open access versionFindings
    • [Codella et al., 2018] Noel CF Codella, David Gutman, M Emre Celebi, Brian Helba, Michael A Marchetti, Stephen W Dusza, Aadi Kalloo, Konstantinos Liopyris, Nabin Mishra, Harald Kittler, et al. Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic). In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pages 168–172. IEEE, 2018.
      Google ScholarLocate open access versionFindings
    • [Finn et al., 2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1126– 1135. JMLR. org, 2017.
      Google ScholarLocate open access versionFindings
    • [Goode et al., 2013] Adam Goode, Benjamin Gilbert, Jan Harkes, Drazen Jukic, and Mahadev Satyanarayanan. Openslide: A vendor-neutral software foundation for digital pathology. Journal of pathology informatics, 4, 2013.
      Google ScholarLocate open access versionFindings
    • [He et al., 2015] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026–1034, 2015.
      Google ScholarLocate open access versionFindings
    • [Ketkar, 2017] Nikhil Ketkar. Introduction to pytorch. In Deep learning with python, pages 195–208.
      Google ScholarFindings
    • [Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
      Findings
    • [Liu et al., 2019] Zechun Liu, Haoyuan Mu, Xiangyu Zhang, Zichao Guo, Xin Yang, Kwang-Ting Cheng, and Jian Sun. Metapruning: Meta learning for automatic neural network channel pruning. In The IEEE International Conference on Computer Vision (ICCV), October 2019.
      Google ScholarLocate open access versionFindings
    • [Munkhdalai and Yu, 2017] Tsendsuren Munkhdalai and Hong Yu. Meta networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 2554–2563. JMLR. org, 2017.
      Google ScholarLocate open access versionFindings
    • [Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computerassisted intervention, pages 234–241.
      Google ScholarLocate open access versionFindings
    • [Tokunaga et al., 2019] Hiroki Tokunaga, Yuki Teramoto, Akihiko Yoshizawa, and Ryoma Bise. Adaptive weighting multi-field-of-view cnn for semantic segmentation in pathology. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 12597– 12606, 2019.
      Google ScholarLocate open access versionFindings
    • [Tschandl et al., 2018] Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data, 5:180161, 2018.
      Google ScholarLocate open access versionFindings
    • [Yu et al., 2018] Changqian Yu, Jingbo Wang, Chao Peng, Changxin Gao, Gang Yu, and Nong Sang. Bisenet: Bilateral segmentation network for real-time semantic segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 325–341, 2018.
      Google ScholarLocate open access versionFindings
    • [Zhao et al., 2017] Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2881– 2890, 2017.
      Google ScholarLocate open access versionFindings
    Your rating :
    0

     

    Tags
    Comments