Automated diagnosis of age-related macular degeneration using multi-modal vertical plane feature fusion via deep learning

Purpose To develop a computer-aided diagnostic (CADx) system of age-related macular degeneration (AMD) through feature fusion between infrared reflectance (IR) and optical coherence tomography (OCT) modalities in order to explore the superiority of multi-modality CADx system and the optimal feature fusion patterns between multi-modality inputs. Methods This is a dual center retrospective study. We retrospectively collected 2006 pairs of IR and OCT images to develop the algorithms. Two single-modality models and three multi-modality models were constructed for the comparison of the diagnostic efficacy. The multi-modality models were designed utilizing a novel feature fusion method, namely, vertical plane feature fusion (VPFF). The results were validated using an independent external validation dataset and compared by three ophthalmologists. Results In the test set of the ZJU dataset, our best model named OCT_MAIN demonstrated diagnostic efficiency with an overall accuracy of 0.9608 and area under the curve of 0.9944 for the normal category, 0.9659 for the dry AMD category, and 0.9930 for the wet AMD category. The external validation exhibited an overall accuracy of 0.9159. Its diagnostic efficiency was comparable to that of the senior ophthalmologist. Conclusions The VPFF method was successfully employed to develop a multi-modal intelligent diagnostic system for the AMD classification. This is a valuable complement and optimization to the existing CADx system, which reveals a wide application prospect and research potential.