Exploring graph capsual network and graphormer for graph classification.

Graph Neural Networks (GNNs) have achieved significant success in many scenarios. However, scalar-based features in GNN may be insufficient for the generation of graph representation via node representation, resulting in a lack of graph properties. Furthermore, the message aggregation operation of the original GNN suffers from not only a lack of global and structural information in graph but also the over-smoothing problem. As a result, we make efforts to alleviate the above problems of the original GNN. Inspired by the idea of capsule, we convert the node features into the form of capsules and utilize dynamic routing mechanism to generate the graph capsules from various perspectives and attention mechanism to highlight important graph capsules. Additionally, to empower the graph capsules with the ability to capture global and structural information in graph and semantic information between nodes, we introduce the Transformer architecture specific to graph data with GraphNorm to enhance the expressiveness of graph capsules and perform the interpretability analysis for the message passing between capsules and the generation process of capsules by visualization technology. Finally, we conduct extensive experiments on six common graph datasets to demonstrate CapsTrans superiority on the graph classification task over existing State-of-the-Art (SOTA) methods.