Validity of inpatient electronic health record‐based measures of oxygen‐related therapy in the United States: Lessons applicable for studying COVID‐19 endpoints

AbstractIntroductionDuring the COVID‐19 pandemic, inpatient electronic health records (EHRs) have been used to conduct public health surveillance and assess treatments and outcomes. Invasive mechanical ventilation (MV) and supplemental oxygen (O2) use are markers of severe illness in hospitalized COVID‐19 patients. In a large US system (n = 142 hospitals), we assessed documentation of MV and O2 use during COVID‐19 hospitalization in administrative data versus nursing documentation.MethodsWe identified 319 553 adult hospitalizations with a COVID‐19 diagnosis, February 2020–October 2022, and extracted coded, administrative data for MV or O2. Separately, we developed classification rules for MV or O2 supplementation from semi‐structured nursing documentation. We assessed MV and O2 supplementation in administrative data versus nursing documentation and calculated ordinal endpoints of decreasing COVID‐19 disease severity. Nursing documentation was considered the gold standard in sensitivity and positive predictive value (PPV) analyses.ResultsIn nursing documentation, the prevalence of MV and O2 supplementation among COVID‐19 hospitalizations was 14% and 75%, respectively. The sensitivity of administrative data was 83% for MV and 41% for O2, with both PPVs above 91%. Concordance between sources was 97% for MV (κ = 0.85), and 54% for O2 (κ = 0.21). For ordinal endpoints, administrative data accurately identified intensive care and MV but underestimated hospitalizations with O2 requirements (42% vs. 18%).ConclusionsIn comparison to nursing documentation, administrative data under‐ascertained O2 supplementation but accurately estimated severe endpoints such as MV. Nursing documentation improved ascertainment of O2 among COVID‐19 hospitalizations and can capture oxygen requirements in adults hospitalized with COVID‐19 or other respiratory illnesses.