A Convolutional Neural Network for Real Time Classification, Identification, and Labelling of Vocal Cord and Tracheal Using Laryngoscopy and Bronchoscopy Video

Background The use of artificial intelligence, including machine learning, is increasing in medicine. Use of machine learning is rising in the prediction of patient outcomes. Machine learning may also be able to enhance and augment anesthesia clinical procedures such as airway management. In this study, we sought to develop a machine learning algorithm that could classify vocal cords and tracheal airway anatomy real-time during video laryngoscopy or bronchoscopy as well as compare the performance of three novel convolutional networks for detecting vocal cords and tracheal rings. Methods Following institutional approval, a clinical dataset of 775 video laryngoscopy and bronchoscopy videos was used. The dataset was divided into two categories for use for training and testing. We used three convolutional neural networks (CNNs): ResNet, Inception and MobileNet. Backpropagation and a mean squared error loss function were used to assess accuracy as well as minimize bias and variance. Following training, we assessed transferability using the generalization error of the CNN, sensitivity and specificity, average confidence error, outliers, overall confidence percentage, and frames per second for live video feeds. After the training was complete, 22 models using 0 to 25,000 steps were generated and compared. Results The overall confidence of classification for the vocal cords and tracheal rings for ResNet, Inception and MobileNet CNNs were as follows: 0.84, 0.78, and 0.64 for vocal cords, respectively, and 0.69, 0.72, 0.54 for tracheal rings, respectively. Transfer learning following additional training resulted in improved accuracy of ResNet and Inception for identifying the vocal cords (with a confidence of 0.96 and 0.93 respectively). The two best performing CNNs, ResNet and Inception, achieved a specificity of 0.985 and 0.971, respectively, and a sensitivity of 0.865 and 0.892, respectively. Inception was able to process the live video feeds at 10 FPS while ResNet processed at 5 FPS. Both were able to pass a feasibility test of identifying vocal cords and tracheal rings in a video feed. Conclusions We report the development and evaluation of a CNN that can identify and classify airway anatomy in real time. This neural network demonstrates high performance. The availability of artificial intelligence may improve airway management and bronchoscopy by helping to identify key anatomy real time. Thus, potentially improving performance and outcomes during these procedures. Further, this technology may theoretically be extended to the settings of airway pathology or airway management in the hands of experienced providers. The researchers in this study are exploring the performance of this neural network in clinical trials.