Predicting Opportunities for Improvement in Trauma using Machine Learning: A Registry Based Study

Rationale The multidisciplinary mortality and morbidity conference is the core of programs that aim to improve the quality of trauma care and is used to identify and address opportunities for improvement based on reviewing patient cases. Current systems rely on audit filters when selecting patients for review, a process that is hampered by high frequencies of false positives. Objectives To develop, validate, and compare the performance of different machine learning models for predicting opportunities for improvement. Methods We conducted a registry based study using all patients from the Karolinska university hospital that had been reviewed regarding the presence of opportunity for improvement, a binary consensus decision from the mortality and morbidity conference. We developed eight binary classification models using 45 predictors. Training used an 80%-20% train-test split and 1000 resamples without replacement estimated confidence intervals. Performance was also compared to current audit filters. Measurements and Main Results The dataset included 6310 patients where opportunities for improvement were present among 431 (7%) patients. The audit filters (Area under the receiver operating characteristics curve: 0.624) was outperformed by all machine learning models. The best performing model was LightGBM (Area under the receiver operating characteristics curve: 0.789). Conclusions Machine learning models outperform the currently used audit filters and could prove to be valuable additions in the screening for opportunities for improvement. More research is needed on how to increase model performance and how to incorporate these models into trauma quality improvement programs.