Enhancing the utilization of deep learning to predict patient response in small immunotherapy cohorts using real-world data

Abstract Real-world evidence (RWE) can provide useful insights to support the design of clinical trials by, for example, helping to identify biomarkers associated with patient response to a given treatment. Because RWE comprises large, diverse, and incomplete data entries (e.g., missing treatments, survival endpoints, response), however, its impact can be limited. We introduce a deep learning AI framework designed to utilize diverse types and qualities of available RWE data, demonstrating its potential application in the clinical setting. The deep learning framework was built on top of transformer architecture to maximize the value of large, unlabeled RWE data for predicting patient survival. The framework has two stages. (1) Pre-training, an oracle model is trained by using the GENIE v.11 data set (N = 136,000), including 2,290 variables via standard masked self-supervised learning. (2) Transfer learning, the oracle’s weights are transferred to a survival model that has the same architecture but outputs a survival score. This model is fine-tuned to predict patient survival and is optimized using the sigmoid approximation of Harrell’s concordance index. The survival model is benchmarked independently across four different immunooncology (IO) and hormone therapy (HT) data sets: pan-cancer (IO, N = 1610), lung cancer from MSK MIND (IO, N = 246) and the MYSTIC trial (IO, N = 325), and melanoma (IO, N = 110) and breast cancer from METABRIC (HT, N = 1981). We built two survival models per data set, with (+) and without (-) transfer learning. Models were evaluated using the concordance index (CI) on 10 training (80%)/testing (20%) splits. Transfer learning from the GENIE data set consistently improved patient survival prediction in almost all the data sets evaluated in this study: pan-cancer data set: from CI(-) = 0.63 ± 0.015 in 86 epochs to CI(+) = 0.65 ± 0.023 in 23 epochs, patient stratification from HR(-) = 0.52 (0.35-0.78) to HR(+) = 0.34 (0.22-0.53; MSK MIND from a CI(-) = 0.56 ± 0.023 in 57 epochs to CI(+) = 0.59 ± 0.031 in 23 epochs; MYSTIC trial: from a CI(-) = 0.56 ± 0.052 in 95 epochs to CI(+) = 0.60 ± 0.05 in 38 epochs; melanoma from a CI(-) = 0.59 ± 0.1 in 85 epochs to CI(+) = 0.63 ± 0.07 in 38 epochs; and METABRIC from a CI(-) = 0.60 ± 0.016 in 62 epochs to CI(+) = 0.59 ± 0.012 in 15 epochs. In addition, the models with transfer learning trained 2.5 times faster. Our framework leverages the value of RWE data sets, such as GENIE, to enhance the utilization of complex modeling approaches for small clinical data sets. In systems where multiple players, such as, host immune system, patient health, and tumor biology are all contributing to patient’s response to treatment, complex modeling such as deep learning is essential to capture these convoluted relationships. We anticipate that our framework has the potential to set the path for improved predictions of patient survival in clinical trials. Citation Format: Gustavo Arango-Argoty, Etai Jacob. Enhancing the utilization of deep learning to predict patient response in small immunotherapy cohorts using real-world data [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 1174.