MRNGAN: Reconstructing 3D MRI Scans Using A Recurrent Generative Model

Oftentimes, physicians find themselves with only a partial magnetic resonance imaging (MRI) scan of a brain, and not the full volume. That is, it is difficult for physicians to obtain an MRI scan of the full volume of a patient’s brain. Instead, physicians may only have a few 2D slices of the full volume, making diagnostics extremely challenging, as it becomes hard to align the scans, and determine where in the brain each 2D slice is. Thus, we present a method for generating the entire volume of the brain given as little as one 2D slice. Another application for this 2D to 3D reconstruction task is to augment and supplement incomplete datasets, which could potentially be very beneficial to deep learning based brain imaging research, as most MRI datasets are quite small. To tackle this 3D reconstruction challenge, we used numerous previously researched topics, such as generative adversarial networks (GANs) and pix2pix, before amalgamating them into our final RNN based model. In this paper we present our novel end-to-end model for reconstructing an MRI scan of the entire volume of a brain from one (or a few) 2D slices and show both quantitatively and qualitatively on a sample dataset that our model creates high fidelity reconstructions.