Deep Multispectral Image Registration Network

Multispectral imaging (MSI) of the ocular fundus provides a sequence of narrow-band images to show the different depths in the retina and choroid. One challenge in analyzing MSI images comes from the image-to-image spatial misalignment, which occurs because the acquisition time of eye MSI images is commonly longer than the natural time scale of the eye's saccadic movement. It is necessary to align images because ophthalmologists usually overlay two of the images to analyze specific features when analyzing MSI images. In this paper, we propose a weakly supervised MSI image registration network, called MSI-R-NET, for multispectral fundus image registration. Compared to other deep-learning-based registration methods, MSI-R-NET utilizes the blood vessel segmentation label to provide spatial correspondence. In addition, we employ a feature equilibrium module to connect the aggregating layers better, and propose a multiresolution auto-context structure to adapt the registration task. In the testing stage, given a new pair of MSI images, the trained model can predict the pixelwise spatial correspondence without labeled blood vessel information. The experimental results demonstrate that the proposed segmentation-driven registration method is highly accurate.