Learning brain representation using recurrent Wasserstein generative adversarial net

Background and objective: To understand brain cognition and disorders, modeling the mapping between mind and brain has been of great interest to the neuroscience community. The key is the brain repre-sentation, including functional brain networks (FBN) and their corresponding temporal features. Recently, it has been proven that deep learning models have superb representation power on functional magnetic resonance imaging (fMRI) over traditional machine learning methods. However, due to the lack of high -quality data and labels, deep learning models tend to suffer from overfitting in the training process. Methods: In this work, we applied a recurrent Wasserstein generative adversarial net (RWGAN) to learn brain representation from volumetric fMRI data. Generative adversarial net (GAN) is widely used in nat-ural image generation and is able to capture the distribution of the input data, which enables the ex-traction of generalized features from fMRI and thus relieves the overfitting issue. The recurrent layers in RWGAN are designed to better model the local temporal features of the fMRI time series. The dis-criminator of RWGAN works as a deep feature extractor. With LASSO regression, the RWGAN model can decompose the fMRI data into temporal features and spatial features (FBNs). Furthermore, the generator of RWGAN can generate high-quality new data for fMRI augmentation. Results: The experimental results on seven tasks from the HCP dataset showed that the RWGAN can learn meaningful and interpretable temporal features and FBNs, compared to HCP task designs and general linear model (GLM) derived networks. Besides, the results on different training datasets showed that the RWGAN performed better on small datasets than other deep learning models. Moreover, we used the generator of RWGAN to yield fake subjects. The result showed that the fake data can also be used to learn meaningful representation compared to those learned from real data. Conclusions: To our best knowledge, this work is among the earliest attempts of applying generative deep learning for modeling fMRI data. The proposed RWGAN offers a novel methodology for learning brain representation from fMRI, and it can generate high-quality fake data for the potential use of fMRI data augmentation. (c) 2022 Published by Elsevier B.V.