Validated machine learning tools to distinguish checkpoint inhibitor, radiotherapy, COVID-19 and other infective pneumonitis

Background Pneumonitis is a well-described, potentially disabling, or fatal adverse effect associated with both immune checkpoint inhibitors (ICI) and thoracic radiotherapy. Accurate differentiation between checkpoint inhibitor pneumonitis (CIP) radiation pneumonitis (RP), and infective pneumonitis (IP) is crucial for swift, appropriate, and tailored management to achieve optimal patient outcomes. However, correct diagnosis is often challenging, owing to overlapping clinical presentations and radiological patterns. Methods In this multi-centre study of 455 patients, we used machine learning with radiomic features extracted from chest CT imaging to develop and validate five models to distinguish CIP and RP from COVID-19, non-COVID-19 infective pneumonitis, and each other. Model performance was compared to that of two radiologists. Results Models to distinguish RP from COVID-19, CIP from COVID-19 and CIP from non-COVID-19 IP out-performed radiologists (test set AUCs of 0.92 vs 0.8 and 0.8; 0.68 vs 0.43 and 0.4; 0.71 vs 0.55 and 0.63 respectively). Models to distinguish RP from non-COVID-19 IP and CIP from RP were not superior to radiologists but demonstrated modest performance, with test set AUCs of 0.81 and 0.8 respectively. The CIP vs RP model performed less well on patients with prior exposure to both ICI and radiotherapy (AUC 0.54), though the radiologists also had difficulty distinguishing this test cohort (AUC values 0.6 and 0.6). Conclusion Our results demonstrate the potential utility of such tools as a second or concurrent reader to support oncologists, radiologists, and chest physicians in cases of diagnostic uncertainty. Further research is required for patients with exposure to both ICI and thoracic radiotherapy.