Development and validation of automated computer aided-risk score for predicting the risk of in-hospital mortality using first electronically recorded blood test results and vital signs for COVID-19 hospital admissions: a retrospective development and validation study

Objectives There are no established mortality risk equations specifically for unplanned emergency medical admissions which include patients with the novel coronavirus SARS-19 (COVID-19). We aim to develop and validate a computer-aided risk score (CARMc19) for predicting mortality risk by combining COVID-19 status, the first electronically recorded blood test results and latest version of the National Early Warning Score (NEWS2). Design Logistic regression model development and validation study using a cohort of unplanned emergency medical admissions to hospital. Setting York Hospital (YH) as model development dataset and Scarborough Hospital (SH) as model validation dataset. Participants Unplanned adult medical admissions discharged over three months (11 March 2020 to 13 June 2020) from two hospitals (YH for model development; SH for external model validation) based on admission NEWS2 electronically recorded within ±24 hours and/or blood test results within ±96 hours of admission. We used logistic regression modelling to predict the risk of in-hospital mortality using two models: 1) CARMc19\_N: age + sex + NEWS2 including subcomponents + COVID19; 2) CARMc19\_NB: CARMc19_N in conjunction with seven blood test results and acute kidney injury score. Model performance was evaluated according to discrimination (c-statistic), calibration (graphically), and clinical usefulness at NEWS2 thresholds of 4+, 5+, 6+. Results The risk of in-hospital mortality following emergency medical admission was similar in development and validation datasets (8.4% vs 8.2%). The c-statistics for predicting mortality for Model CARMc19\_NB is better than CARMc19\_N in the validation dataset (CARMc19\_NB = 0.88 (95%CI 0.86 to 0.90) vs CARMc19\_N = 0.86 (95%CI 0.83 to 0.88)). Both models had good internal and external calibration (CARMc19\_NB: 1.01 (95%CI 0.88 vs 1.14) & CARMc19\_N: 0.95 (95%CI 0.83 to 1.06)). At all NEWS2 thresholds (4+, 5+, 6+) model CARMc19_NB had better sensitivity and similar specificity. Conclusions We have developed a validated CARMc19 score with good performance characteristics for predicting the risk of in-hospital mortality following an emergency medical admission using the patient’s first, electronically recorded vital signs and blood tests results. Since the CARMc19 scores place no additional data collection burden on clinicians and is readily automated, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure. ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement This research was supported by the Health Foundation. The Health Foundation is an independent charity working to improve the quality of healthcare in the UK. This research was also supported by the National Institute for Health Research (NIHR) Yorkshire and Humber Patient Safety Translational Research Centre (NIHR Yorkshire and Humber PSTRC). The views expressed in this article are those of the author(s) and not necessarily those of the NHS, the Health Foundation, the NIHR, or the Department of Health. ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: This study was deemed to be exempt from ethical approval because it was classified as an evaluation. Furthermore, this study used already de-identified data from an ongoing study involving NEWS which received ethical approval from Health Research Authority (HRA) and Health and Care Research Wales (HCRW) (reference number 19/HRA/0548). All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable. Yes Our data sharing agreement is with York hospital and does not permit us to share the data used in this paper.