Triage and diagnostic accuracy of Online Symptom Checkers: a systematic review (Preprint)

BACKGROUND In the context of a deepening global shortage of health workers, and particularly the COVID-19 pandemic, there is growing international interest in and use of online symptom checkers (OSCs). However, the evidence surrounding the safety and accuracy of OSCs remains inconclusive so far. The triage and diagnostic accuracy of these tools is an essential aspect that needs to be addressed before pushing any further implementation. OBJECTIVE This systematic review aimed to summarize the existing peer-reviewed literature evaluating the triage accuracy (directing users to appropriate services based on their presenting symptoms) and diagnostic accuracy of OSCs aimed at lay users for general health concerns. METHODS Searches were conducted in Medline, Embase, CINAHL, HMIC and Web of Science. We included peer-reviewed studies published in English between 1 January 2010 and 17 February 2022 with a quantitative assessment of triage and/or diagnostic accuracy of OSCs directed at lay users. We excluded tools supporting health professionals, and disease- or speciality-specific OSCs. Screening and data extraction were carried out independently by two reviewers for each study. We performed a descriptive narrative synthesis. RESULTS 21,284 studies were screened and 15 were included. Six studies reported on both triage and diagnostic accuracy, eight focused on triage accuracy, and one on diagnostic accuracy. Diagnostic and triage accuracy varied between studies and OSCs; most studies showed suboptimal diagnostic and triage accuracy. Frequency and urgency of the condition were the main variables that affected the levels of diagnostic and triage accuracy, along with specific features of the OSCs. The impact of each variable differed across tools and studies, making it difficult to draw any solid conclusions. Included studies had either a moderate or high risk of bias according to the revised tool for the Quality Assessment of Diagnostic Accuracy Studies 2. CONCLUSIONS While OSCs have significant potential to provide accessible and accurate health advice and triage recommendations to users, more research is needed to validate their triage and diagnostic accuracy prior to wide scale adoption in community and healthcare settings. Future studies should aim to use a common methodology and/or agreed standard for evaluation to facilitate objective benchmarking and validation.