A Time-Incorporated SOFA-Based Explainable Machine Learning Model for Mortality Prediction in Critically Ill Patients

Background: Organ dysfunction (OD) assessment is essential in intensive care units (ICUs). However, no OD scoring system has so far considered the duration of OD, which is clinically relevant. This study aimed to develop and validate an ICU mortality prediction model based on the Sequential Organ Failure Assessment (SOFA) score, incorporating the time dimension with machine learning methods.Methods: Data from the eICU Collaborative Research Database and Medical Information Mart for Intensive Care (MIMIC) -III were mixed for model development, and the MIMIC-IV dataset and Nanjing Jinling Hospital Surgery ICU (SICU-JL) dataset were used for external testing. Adult patients in the ICUs for more than 72 hours were deemed eligible. The total SOFA score and individual scores were calculated every 12 hours for the first three days of ICU admission. Time-dimensional variables were derived from the consecutively recorded SOFA scores and individual scores for each organ. A modified SOFA model incorporating the time dimension (T-SOFA) was stepwise constructed to predict ICU mortality using multiple machine learning algorithms. The predictive performance was assessed with the area under the receiver operating characteristic curves (AUROC). Also, we utilized the SHapley Additive exPlanations (SHAP) algorithm for data visualization and model explainability.Results: We extracted a total of 66,709 ICU patients from the mixed datasets for model development and 15,423 patients for validation. The T-SOFA M3 that incorporated the time dimension features and age, using the XGBoost algorithm, significantly outperformed the original SOFA scores (AUROC 0.800 95% CI [0.787-0.813] vs. 0.693 95% CI [0.678-0.709], p<0.01) in the validation set. Good prediction performance was maintained for the T-SOFA M3 in test Set A and test Set B, with AUROC of 0.803, 95% CI[0.791-0.815], and 0.830, 95%CI [0.789-0.870], respectively. Significant contributors demonstrated by the SHAP analysis included total SOFA score, Respiration-score, CNS-score, age, Cardiovascular-score, and SOFA Organ dysfunction Unalleviated Time Index.Conclusions: A SOFA-based, time-incorporated prediction model was developed and validated by machine learning algorithms, showing satisfactory predictability and medical interpretability.