Visual foundation models for fast, label-free detection of diffuse glioma infiltration

Abstract A critical challenge in diffuse glioma treatment is detecting tumor infiltration during surgery to achieve safe maximal resection[1-3]. Unfortunately, safely resectable residual tumor is found in the majority of glioma patients after surgery, leading to early recurrence and decreased patient survival[4-6]. We present FastGlioma, a visual foundation model for fast (<10 seconds) and accurate detection of glioma infiltration in fresh, unprocessed surgical tissue. FastGlioma was pretrained using large-scale self-supervision (~4 million images) on rapid, label-free, optical microscopy, and fine-tuned to output a normalized score that indicates the degree of tumor infiltration within whole slide optical images. In a prospective, multicenter, international testing cohort of diffuse glioma patients (n=220), FastGlioma was able to detect and quantify the degree of tumor infiltration with an average area under the ROC curve of 92.1 ± 0.9%. FastGlioma outperformed image-guided and fluorescence-guided adjuncts for detecting tumor infiltration during surgery by a wide margin in a head-to-head, prospective comparison study (n=129). FastGlioma performance remained high across diverse patient demographics, medical centers, and diffuse glioma molecular subtypes as defined by the World Health Organization (WHO). FastGlioma shows zero-shot generalization to other adult and pediatric brain tumor diagnoses, demonstrating that our visual foundation model can serve as a general-purpose adjunct for guiding brain tumor surgeries. These findings represent the transformative potential of medical foundation models to unlock the role of artificial intelligence in the care of cancer patients.