Automated deep neural network-based identification, localization, and tracking of cardiac structures for ultrasound-guided interventional surgery

Background: The increase in the use of ultrasound-guided interventional therapy for cardiovascular diseases has increased the importance of intraoperative real-time cardiac ultrasound image interpretation. We thus aimed to develop a deep learning-based model to accurately identify, localize, and track the critical cardiac structures and lesions (9 kinds in total) and to validate the algorithm's performance using independent data sets.Methods: This diagnostic study developed a deep learning-based model using data collected from Fuwai Hospital between January 2018 and June 2019. The model was validated with independent French and American data sets. In total, 17,114 cardiac structures and lesions were used to develop the algorithm. The model findings were compared with those of 15 specialized physicians in multiple centers. For external validation, 516,805 tags and 27,938 tags were used from 2 different data sets.Results: Regarding structure identification, the area under the receiver operating characteristic curve (AUC) of each structure in the training data set, optimal performance in the test data set, and median AUC of each structure identification were 1 (95% CI: 1-1), 1 (95% CI: 1-1), and 1 (95% CI: 1-1), respectively. Regarding structure localization, the optimal average accuracy was 0.83. As for structure identification, the accuracy of the model significantly outperformed the median performance of the experts (P<0.01). The optimal identification accuracies of the model in 2 independent external data sets were 89.5% and 90%, respectively (P=0.626).Conclusions: The model outperformed most human experts and was comparable to the optimal performance of all human experts in cardiac structure identification and localization, and could be used in the external data sets.