Comprehensive Quantitative Evaluation of Inter-observer Delineation Performance of MR-guided Delineation of Oropharyngeal Gross Tumor Volumes and High-risk Clinical Target Therapy: An R-IDEAL Stage 0 Prospective Study

Purpose Tumor and target volume manual delineation remains a challenging task in head and neck cancer radiotherapy. The purpose of this study is to conduct a multi-institutional evaluation of manual delineations of gross tumor volume (GTV), high-risk clinical target volume (CTV), parotids, and submandibular glands on treatment simulation MR scans of oropharyngeal cancer (OPC) patients. Methods Pre-treatment T1-weighted (T1w), T1-weighted with Gadolinium contrast (T1w+C) and T2-weigted (T2w) MRI scans were retrospectively collected for 4 OPC patients under an IRB-approved protocol. The scans were provided to twenty-six radiation oncologists from seven international cancer centers who participated in this delineation study. In addition, each patient’s clinical history and physical examination findings along with a medical photographic image and radiological results were provided. The contours were compared using overlap and distance metrics using both STAPLE and pair-wise comparisons. Lastly, participants completed a brief questionnaire to assess personal experience and CTV delineation institutional practices. Results Large variability was measured between observers’ delineations for both GTVs and CTVs. The mean Dice Similarity Coefficient values across all patients’ delineations for GTVp, GTVn, CTVp, and CTVn where 0.77, 0.67, 0.77, and 0.69, respectively, for STAPLE comparison and 0.67, 0.60, 0.67, and 0.58, respectively, for pair-wise analysis. Normal tissue contours were defined more consistently when considering overlap and distance metrics. The median radiation oncology clinical experience was 7 years and the median experience delineating on MRI was 3.5 years. The GTV-to-CTV margin used was 10 mm for six of seven participant institutions. One institution used 8 mm and three delineators (from three different institutions) used a margin of 5 mm. Conclusion The data from this study suggests that appropriate guidelines, contouring quality assurance sessions, and training are still needed for the adoption of MR-based treatment planning for head and neck cancers. Such efforts should play a critical role in reducing inter-observer delineation variation and ensure standardization of target design across clinical practices. Summary With the rapid increase of MR imaging use in radiotherapy, MRI presents with endogenous contrast greater than CT for tumor and target definition; however, reports on delineation variability for these volumes on MRI are limited. In this prospective manual segmentation challenge of 26 radiation oncologists, we formally quantify human performance and heterogeneity in physician tumor and target delineations based on MRI scans of oropharyngeal cancer patients. ### Competing Interest Statement Funding sources: Infrastructure and in-kind support was provided via the MR-LinAc Consortium. Direct support for this project was provided to MD Anderson Cancer Center and Elekta AB from the National Institute for Dental and Craniofacial Research Academic-Industrial Partnership Award (R01-DE028290, PI Fuller). Dr. Fuller has received direct industry grant support and travel funding from Elekta AB. ### Funding Statement This work was supported by infrastructure support from the MR-LinAc Consortium. Dr. Fuller has received direct industry grant support and travel funding from Elekta AB. This research was supported by the Andrew Sabin Family Foundation; Dr. Fuller was a 2017-2019 a Sabin Family Foundation Fellow. Dr. Fuller received funding and salary support during study execution interval from the National Institutes of Health (NIH), including: the National Institute for Dental and Craniofacial Research (R01DE028290;1R01DE025248/R56DE025248); a National Science Foundation (NSF), Division of Mathematical Sciences, Joint NIH/NSF Initiative on Quantitative Approaches to Biomedical Big Data (QuBBD) Grant (NSF 1557679); a National Institute of Biomedical Imaging and Bioengineering (NIBIB) Research Education Programs for Residents and Clinical Fellows Grant (R25EB025787-01); the NIH Big Data to Knowledge (BD2K) Program of the National Cancer Institute (NCI) Early Stage Development of Technologies in Biomedical Computing, Informatics, and Big Data Science Award (1R01CA214825); NCI Early Phase Clinical Trials in Imaging and Image-Guided Interventions Program (1R01CA218148); an NIH/NCI Cancer Center Support Grant (CCSG) Pilot Research Program Award from the UT MD Anderson CCSG Radiation Oncology and Cancer Imaging Program (P30CA016672) and an NIH/NCI Head and Neck Specialized Programs of Research Excellence (SPORE) Developmental Research Program Award (P50 CA097007). ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: Ethics Committee/IRB of University Medical Center-Utrecht gave ethical approval for anonymized use for this study. I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable. Yes Anonymized data are embargoed until peer-review completion, but thereafter available at