Optimization of artificial intelligence algorithms for low-resource/clinical environments: focus on clinically-relevant glioma region delineation

Abstract BACKGROUND Diffuse astrocytic glioma are common and aggressive malignant primary brain tumors with grim prognosis. Artificial intelligence (AI) has shown promise across predictive, prognostic, and diagnostic neuro-oncology applications, towards improving patient management. However, clinical translation and deployment are hampered by AI models’ requirements for explicit acceleration cards, which are not typically considered in clinical environments. Here, we seek the execution of AI models in such clinical/low-resource environments, facilitated by mathematical optimizations rather than investing in acceleration cards, and focus on the use case of delineating clinically-relevant regions, namely the entire tumor burden (ETB) and tumor core (TC). METHODS We identified the BraTS20 retrospective cohort of 369 glioma cases, each described by 4 structural multi-parametric MRI (mpMRI) scans (T1,T1Gd,T2,T2-FLAIR). The Generally Nuanced Deep Learning Framework (GaNDLFv0.0.14/PyTorchv1.8.2) was used to train the original AI model (ResUNet), by randomly sampling 40 643 patches from each mpMRI scan. We then investigate the contributions of post-training mathematical optimizations. Quantitative performance evaluation of the original and optimized (GaNDLFv0.0.14/OpenVINOTM-INT8-v2022.1.0) models were based on 125 unseen hold-out cases (BraTS20 validation dataset), using the dice similarity coefficient (DSC), while profiling in a consumer-grade workstation, i.e., typical clinical hardware configuration. RESULTS Negligible delineation performance differences were observed between the original and optimized AI models, for both ETB (DSCoriginal/DSCoptimized= 0.877/0.876) and TC (DSCoriginal/DSCoptimized= 0.773/0.772). However, the optimized model yielded substantial improvements in latency (up to 5.4x faster inference) and 53% less memory footprint. CONCLUSIONS Post-training mathematical optimization of AI models yields substantial gains in latency and memory usage, while maintaining their accuracy. Although we focused on delineation, we anticipate these mathematical optimizations to be applicable in other AI models. Post-training optimization is a promising approach for deploying AI models on consumer-grade CPUs, and hence facilitating their translation in low-resource/clinical environments, potentially contributing to improved patient management, treatment decisions, and response assessment. *author Siddhesh Thakur is an equal contributing first author.