56: external validation of pet-based radiomic models to identify patients with residual esophageal cancer after neoadjuvant chemoradiotherapy

Abstract Background and aim High-throughput quantitative imaging (‘radiomics’) has been proposed to predict tumor response in various types of cancer. Internally validated radiomic models based on post-treatment PET features plus cT-stage have been developed (Beukinga et al. 2018) to detect residual tumor after neoadjuvant chemoradiotherapy for esophageal cancer. The aim of the present study was to externally validate the published models. Methods The external validation cohort comprised esophageal cancer patients who underwent chemoradiotherapy according to the CROSS regimen followed by immediate resection in four Dutch institutes between 2013–2019. Outcome was tumor regression grade (TRG) 1 (0% residual vital tumor) versus TRG-2-3-4 (≥ 1% tumor). Preoperative FDG-PET/CT was performed 6–12 weeks after nCRT. Gross tumor volumes on CT were transposed to post-treatment PET scans and were manually adapted in consensus by two investigators. Radiomic features were extracted using the open-source software ‘Pyradiomics’, with settings similar to the published models. Discrimination and calibration were assessed for the 6 internally validated models with optimism-corrected AUCs>0.77. Results Some 189 patients were included in the external validation cohort. Baseline characteristics and outcome (TRG-1: 40/189 patients (21%); TRG2–3-4: 149/189 patients (79%)) were comparable to the derivation sample. Since features were dependent on different scanner manufacturers (data not shown), the cohort was reduced with only scans from Siemens scanners (n = 130), similar to the derivation sample. The model including cT-stage and the grey-level-co-occurrence-matrix feature ‘Joint Energy’ had highest AUC of 0.71 (95% CI 0.60–0.81), and a calibration slope and intercept of −0.19 and 2.83 respectively. At a probability threshold of 0.5, sensitivity was 81%, specificity 31% and balanced accuracy 56%. Conclusion The predictive performance of the previously developed radiomic models could not be replicated in the present multicenter external validation cohort. The decreased overall performance indicates overfitting of the models to the derivation sample. Possible causes are unintentional dependency of radiomic features on scanner types, scanning protocols, tumor delineation methods and timing of post-treatment FDG-PET/CT. We aim at improving the generalizability of the models, to enable future application in clinical decision-making for individual patients.