Part-Wise Topology Graph Convolutional Network for Skeleton-Based Action Recognition

Action recognition based on skeleton data has attracted extensive attention in computer vision. Graph convolutional network (GCN) has achieved remarkable performance by modeling the human skeleton as a spatial-temporal graph. The graph topology that dominates feature aggregation is the key for GCN to extract representative features. However, the previous models based on GCN mostly build skeleton topology that are naturally connected or adaptively shared, and lack the exploration of fine-grained relations of multi-level features. In this paper, we propose a novel Part-wise Topology Graph Convolution (PT-GC) for the task of skeleton action recognition. PT-GC first builds part-level topology with two modeling strategies, and then effectively aggregates multi-level joint features by combining global topology and part-level topology, which can accurately construct human topology. Finally, we adopt the two-stream architecture and combine PT-GC with a spatial-temporal modeling module to propose a powerful graph convolutional network named PT-GCN. On the two large-scale datasets, NTU RGB+D and NTU RGB+D 120, PT-GCN exhibits significant performance advantages, proving the effectiveness of our proposed method.