Evaluation and Validation of Artificial Intelligence (AI) Delineation Models for CBCT-guided Stereotactic Online Adaptive Radiotherapy (STAR) Targeting Upper Abdominal Malignancies

An online adaptive radiotherapy platform coupled with a ring gantry Linac was recently released with integrated AI models to assist the delineations of organ-at-risks due to daily anatomy changes. Here we evaluate the efficiency and accuracy of AI auto-segmentations via a prospective in silico CBCT-guided STAR (CT-STAR) trial targeting upper abdominal malignancies. Five patients with upper abdominal malignancies (3 pancreatic, 1 liver and one oligometastatic lymph node) previously treated were assessed in this CT-STAR trial (50Gy in 5 fractions). The patients were planned on CT images and simulated for adaptive treatment based on high quality daily CBCT images acquired on a ring-gantry Linac. AI models of liver, duodenum and stomach are currently available and the contours were automatically generated for each adaptive fraction. Physician and physicist teamed up to comprehensively evaluate and modify those contours to fully mimic re-delineations required in adaptive treatment. Each AI organ contour was ranked by major (>1 min) or minor correction (< 1 min) required based on physician’s online evaluation. AI contours and the physician modified contours were then compared offline. Dice similarity coefficients (DSC) and the average distance from the ground-truth contour were used for accuracy analysis. The time taken to generate AI contours were also recorded to evaluate AI delineation efficiency. A total of 23 fractions were simulated with 61 AI contours (23 liver, 15 stomach and 23 duodenum) generated and evaluated. The average time required to complete the AI auto-contours of all three organs are 46.8±15s. Minor editing was required on 65% liver, 17% stomach and 8.7% duodenum AI contours. The post analysis indicated the mean DSC is 0.93±0.06 (median 0.94), 0.80±0.13 (median 0.85), 0.70 ± 0.18 (median 0.74) for liver, stomach and duodenum, respectively. The average distance between AI and the ground-truth contours are 1.42±1.56mm (median 0.71 mm), 2.53±1.54mm (median 2.08) and 3.52±1.97mm (median 3.43mm) for liver, stomach and duodenum. These results indicate the current AI models are more accurate for liver and stomach, but suboptimal for duodenum. AI contours are generated with high efficiency and have the potential to facilitate online delineation for adaptive therapy. The current contour accuracy is organ-dependent and a thorough verification by the clinical team is critical.