Analyzing Unstructured Clinical Notes For Phase Iv Drug Safety Surveillance

Background: The current state of the art in post-marketing drug surveillance utilizes large collections of submitted reports to detect adverse drug reactions. However, given the limitations of reporting systems, there is an opportunity to meaningfully use electronic health records (EHRs) for next-generation signal detection and advancement of drug safety surveillance. Methods and Results: We present data mining methods that transform unstructured patient notes taken by doctors, nurses and other clinicians into a de-identified, temporally ordered, patient-feature matrix using standardized medical terminologies. We demonstrate how to use the resulting high-throughput data to monitor for adverse drug events based on the clinical notes in the EHR. We analyze the patterns of mentions of disease conditions, drug names, their co-mentions and the temporal ordering of the drugs and diseases, in the output of our text processing pipeline to detect known associations between drugs and their adverse effects. Overall, our methods provide highly accurate results (72% Sensitivity, 83% Specificity) as measured by testing a set of 25 known drug recalls and a set of 200 “true negative” drug-adverse event associations. We are able to detect associations between a drug and the adverse event that results in its recall, on average, 2 years ahead of the official notification. We also describe methods for investigating a suspect drug-adverse event association using stratification, propensity score matching and show that matching based on co-morbidities and co-prescriptions may correct for confounding from unobserved variables; thus making the data-mining methods robust against confounding. Conclusion: We conclude that data-mining of unstructured clinical notes via ontology driven methods can enable meaningful use of the EHR for post-marketing drug surveillance. Such data-mining can be used for hypothesis generation and for rapid retrospective analysis of suspected adverse event risk.