Unified Representation Learning for Efficient Medical Image Analysis

Medical image analysis typically includes several tasks such as image enhancement, detection, segmentation, and classification. These tasks are often implemented through separate machine learning methods, or recently through deep learning methods. We propose a novel multitask deep learning-based approach, called unified representation (U-Rep), that can be used to simultaneously perform several medical image analysis tasks. U-Rep is modality-specific and takes into consideration inter-task relationships. The proposed U-Rep can be trained using unlabeled data or limited amounts of labeled data. The trained U-Rep is then shared to simultaneously learn key tasks in medical image analysis, such as segmentation, classification and visual assessment. We also show that pre-processing operations, such as noise reduction and image enhancement, can be learned while constructing U-Rep. Our experimental results, on two medical image datasets, show that U-Rep improves generalization, and decreases resource utilization and training time while preventing unnecessary repetitions of building task-specific models in isolation. We believe that the proposed method (U-Rep) would tread a path toward promising future research in medical image analysis, especially for tasks with unlabeled data or limited amounts of labeled data.