A Collaborative Fusion of Vision Transformers and Convolutional Neural Networks in Classifying Cervical Vertebrae Maturation Stages.

We present MultiMixer, an innovative deep-learning approach for automating the classification of Cervical Vertebrae Maturation (CVM) stages. Our method is a four-channel network combining three deep Convolutional Neural Networks (CNNs) and a Vision Transformer (ViT) in a parallel architecture. The combination leverages the CNNs’ proficiency in extracting local features and ViT’s ability to capture global dependencies. Each module is trained independently with distinct initialization parameters. Our approach utilizes a set of directional filters to highlight cervical vertebrae edges in X-ray images and the outputs of the directional filters are fed as input to the CNNs ResNet18, DenseNet, and PyramidNets. and the ViT. The subnetwork outputs are fused through a fully connected layer. For training, we annotated 1018 cephalometric radiographs, categorized by gender and classified by the CVM stages. We employed various training techniques, including image patches and adjustable directional edge enhancers, alongside data augmentation methods to mitigate overfitting. In our experiments, MultiMixer achieved an accuracy of 82.35% for female patients and 77.88% for male patients. Notably, our approach outperformed Vision Transformers and other previously studied network models when estimating the CVM stages within our dataset.