Using Natural Language Processing To Investigate Diagnostic Error In Acute Stroke

Introduction: Diagnostic error occurs in approximately 10% of acute stroke (AS) presentations. The diagnostic process includes history, physical examination, and test performance and interpretation. However, critical information for diagnosis is contained in unstructured clinical notes. Hypothesis: We hypothesized that natural language processing (NLP) can identify features in unstructured clinical notes associated with potential diagnostic error during ED “catch and release” (CR) encounters prior to AS admissions. Methods: Using a retrospective case-control design and ICD-10 codes, we identified index emergency department (ED) admissions with a diagnosis of first-time stroke (cases) and age and sex-matched gastroenteritis (controls) who had an ED CR encounter in prior 30 days. Notes were processed using cTAKES to identify concept unique identifiers (CUI) among clinical narratives from the CR encounters. Regression analysis was utilized to determine CUI terms from the CR encounter that were associated with stroke cases compared to controls. These CUI terms were grouped by clinical experts into 3 aspects of the diagnostic process: history (e.g., risk factors, medications, symptoms), neurologic examination (e.g., mental status exam, cranial nerves, pronator drift), and tests (e.g., labs, CT, MRI). Results: In an analytic cohort of 319 stroke cases and 319 gastroenteritis controls, a non-cerebrovascular neurologic diagnosis at the CR encounter was noted in 20.2% of cases versus 6.0% in controls (P<0.01). We identified 120 terms at the CR encounter associated with stroke (OR >2.0 and p<0.05). Grouped by themes, tests accounted for 50 (41.7%), examination for 37 (30.1%), and history for 33 (27.5%) terms. Terms related to neurologic examination had the highest median OR (median OR 6.7, IQR 2.7-11.5) followed by history (median OR 3.8, IQR 3.2-4.9) and tests (median OR 3.5, IQR 2.8-4.6). Conclusions: Neurologic presentations to the ED preceded 20% of stroke cases suggesting some of these may represent missed diagnoses for minor stroke and TIA. NLP may be a useful surveillance approach to identify neurologic symptoms, deficits, and tests present at CR encounters and trigger interventions to reduce diagnostic error prior to stroke.