Connecting quantity and quality: An innovative statistical method for linking ACGME case logs and surgical resident autonomy

Purpose It is unknown how many operative experiences a surgical trainee needs to experience before achieving meaningful autonomy for a given procedure. To answer this question, the number of operative experiences a resident has participated in must be combined with the information about the quality of one or more experiences. This paper demonstrates an innovative statistical method for linking case log data (quantity) and workplace-based assessment (WBA) data (quality). Methods With data from a single general surgery residency program, we linked case log and WBA data using Bayesian generalized linear mixed effects models and marginalized predictions to produce autonomy curves for two procedures: laparoscopic appendectomy and laparoscopic cholecystectomy. Results Autonomy curves accounted for multiple sources of variation affecting whether a trainee was granted meaningful autonomy. The statistical linking approach in this paper had strong predictive validity for a laparoscopic appendectomy autonomy curve (AUC = 0.82) and a laparoscopic cholecystectomy autonomy curve (AUC = 0.86). Conclusions Case log (quantity) and WBA (quality) data were combined and analyzed with mixed effects models and marginalized predictions (linking method) to develop performance curves that were reflective of a sample of general surgery residents’ actual operative experience. This approach could be used to establish case count guidelines for programs and regulators.