Agreement and consistency in the triaging of musculoskeletal primary care referrals by vetting clinicians using a knowledge-based triage tool

Background: Primary care referrals received by secondary care services are vetted or triaged to pathways best suited for patients' needs. If knowledge-based triaging is used by vetting clinicians, accuracy is required to avoid incorrect decisions being made. With limited evidence to support best practice, we aimed to evaluate consistency across vetting clinicians' decisions and their agreement with a criterion decision.Methods: Twenty-nine trained vetting clinicians (18 female) representative of pay grades independently triaged five musculoskeletal physiotherapy referral cases into one of 10 decisions using an internally developed triage tool. Agreement across clinicians' decisions between and within cases was assessed using Fleiss's kappa overall and within pay grade. Proportions of triage decisions consistent with criterion decisions were assessed using Cochran's Q test. Results: Clinician agreement was fair for all cases (kappa = 0.385) irrespective of pay grade but varied within clinical cases (kappa = -0.014-0.786). Proportions of correct triage decisions were significantly different across cases [Q(4) = 33.80, P < 0.001] ranging from 17% to 83%. Conclusions: Agreement and consistency in decisions were variable using the tool. Ensuring referrer information is accurate is vital, as is developing, automating and auditing standards for certain referrals with clear pathways. But we argue that variable vetting outcomes might represent healthy pathway abundance and should not simply be automated in response to perceived inefficiencies.