TransG-net: transformer and graph neural network based multi-modal data fusion network for molecular properties prediction

Molecular properties prediction is an important task in the field of materials, especially in computational drug and materials discovery. Deep learning (DL) is one of the most popular methods for molecular properties prediction due to its ability to establish quantitative relationships between molecular representations and target properties. In order to improve the performance of DL algorithms, it is crucial to select appropriate representation of molecules. Molecular graph has become one of the choices as it can be easily input into graph neural network (GNN)-based DL models for learning. However, model performance is limited if molecular representation is only used because it only contains atomic information, bond information, and adjacency relationships between atoms. Therefore, we use molecular mass spectrum as another representation to provide supplement information which is not contained in the graph data. In this paper, a transformer-based model, named Mass Spectrum Transformer (MST), is proposed to perform quantitative analysis of molecular spectra, then it is combined with the graph neural network to form a multi-modal data fusion model TransG-Net for accurate molecular properties prediction. Several feature fusion methods are adopted and the best method is chosen to further enhance the performance of the model. A multi-modal dataset is collected in this paper which is composed of molecular graph data and spectra. Data augmentation is performed to simulate the experimentally measured molecular spectra for the generalizability of the model. Experimental results show that MST outperforms previous best mass spectrum-based methods for molecular properties prediction. In addition, TransG-Net combining MST and GNN achieves better performance than state-of-the-art well-designed message passing models, which proves the effectiveness of our multi-modal data fusion method.