Exploring the applicability of using natural language processing to support nationwide venous thromboembolism surveillance (Preprint)

BACKGROUND Conducting public health surveillance for venous thromboembolism (VTE) at a national scale is important for measuring the disease burden and the impact of prevention measures. Integrating natural language processing (NLP) into VTE surveillance may be an efficient and accurate option in establishing a sustainable and cost-effective national surveillance system. OBJECTIVE We evaluated the performance of the VTE identification instance of IDEAL-X, an NLP tool, in automatically classifying cases of VTE from “reading” unstructured text from diagnostic imaging records. METHODS Accessing imaging records from pilot surveillance systems for VTE from Duke University and the University of Oklahoma Health Sciences Center (OUHSC) during 2012–2014, we used a VTE identification model of IDEAL-X to classify cases of VTE that had previously been manually classified according to pre-defined criteria. The performance measures (and 95% confidence intervals [CI]) calculated were accuracy, sensitivity, specificity, and positive and negative predictive values. RESULTS The VTE model of IDEAL-X “read” 1591 records from Duke University and 1487 records from OUHSC for a total of 3078 records. The combined performance measures were 93.7% accuracy (95% CI: 93.7%–93.8%), 96.3% sensitivity (95% CI: 96.2%–96.4%), 92.0% specificity (95% CI: 91.9%–92.0%), 89.1% positive predictive value (95% CI: 89.0%–89.2%), and 97.3% negative predictive value (95% CI: 97.3%–97.4%). The sensitivity was higher at Duke University (97.9%, 95% CI: 97.8%–98.0%) than at OUHSC (93.8%, 95% CI: 93.5%–94.0%), but the specificity was higher at OUHSC (96.3%, 95% CI: 96.2%–96.3%) than at Duke University (90.4%, 95% CI: 90.1%–90.5%). CONCLUSIONS The VTE model of IDEAL-X accurately classified cases of VTE from pilot surveillance systems from 2 separate states and health systems. NLP is a promising tool in the design and implementation of an automated national surveillance system for VTE.