Usefulness of an artificial intelligence system for the detection of esophageal squamous cell carcinoma evaluated with videos simulating overlooking situation

Objectives Artificial intelligence (AI) systems have shown favorable performance in the detection of esophageal squamous cell carcinoma (ESCC). However, previous studies were limited by the quality of their validation methods. In this study, we evaluated the performance of an AI system with videos simulating situations in which ESCC has been overlooked. Methods We used 17,336 images from 1376 superficial ESCCs and 1461 images from 196 noncancerous and normal esophagi to construct the AI system. To record validation videos, the endoscope was passed through the esophagus at a constant speed without focusing on the lesion to simulate situations in which ESCC has been missed. Validation videos were evaluated by the AI system and 21 endoscopists. Results We prepared 100 video datasets, including 50 superficial ESCCs, 22 noncancerous lesions, and 28 normal esophagi. The AI system had sensitivity of 85.7% (54 of 63 ESCCs) and specificity of 40%. Initial evaluation by endoscopists conducted with plain video (without AI support) had average sensitivity of 75.0% (47.3 of 63 ESCC) and specificity of 91.4%. Subsequent evaluation by endoscopists was conducted with AI assistance, which improved their sensitivity to 77.7% (P = 0.00696) without changing their specificity (91.6%, P = 0.756). Conclusions Our AI system had high sensitivity for the detection of ESCC. As a support tool, the system has the potential to enhance detection of ESCC without reducing specificity. (UMIN000039645)