Pd30-05 ai-based video feedback to improve novice performance on a robotic suturing task

You have accessJournal of UrologyCME1 Apr 2023PD30-05 AI-BASED VIDEO FEEDBACK TO IMPROVE NOVICE PERFORMANCE ON A ROBOTIC SUTURING TASK Jasper A. Laca, Dani Kiyasseh, Rafal Kocielnik, Taseen F. Haque, Runzhuo Ma, Anima Anandkumar, and Andrew J. Hung Jasper A. LacaJasper A. Laca More articles by this author , Dani KiyassehDani Kiyasseh More articles by this author , Rafal KocielnikRafal Kocielnik More articles by this author , Taseen F. HaqueTaseen F. Haque More articles by this author , Runzhuo MaRunzhuo Ma More articles by this author , Anima AnandkumarAnima Anandkumar More articles by this author , and Andrew J. HungAndrew J. Hung More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000003316.05AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Automated skills assessment can provide surgical trainees with objective, personalized feedback during training. Here, we measure the efficacy of artificial-intelligence (AI)-based feedback on a robotic suturing task. METHODS: 42 participants (no surgical experience) were randomized to control group (CG) or feedback group (FG) and video-recorded while completing 2 rounds (R1 and R2) of suturing tasks on a daVinci surgical robot. Participants were assessed on needle handling (NH) and needle driving (ND), and feedback was provided via a visual interface after R1 (Fig). For CG: participants were presented with randomly-selected video clips from R1. For FG: participants were informed of their AI-based skill assessment and presented with specific video clips from R1. Following the feedback, participants completed R2 where they repeated the suturing task. The ground-truth annotation of skill assessments of all suturing tasks was performed retrospectively by 3 blinded raters using a previously-validated tool (End-To-End Assessment of Suturing Expertise (EASE)). These ground-truth annotations were compared to AI-based assessments. Participants from each group were further labeled as under-performers (UP) or innate performers (IP), based on a median split of EASE scores from R1. Generalized estimating equation was used to compare performance between groups. RESULTS: Human raters achieved intraclass correlation coefficients of 0.84 for manual assessments (p<0.001). AI scoring achieved an AUC of 0.67 and 0.80 for NH and ND, respectively, using human scores as ground truth. By AI scores: R1 performance was statistically similar between CG (n=20) and FG (n=22) (p>0.05). Observing the median difference across rounds (R1 vs. R2), FG had a significantly larger improvement in NH skill (0.21 vs 0, p=0.032) and a larger improvement in ND skill (0.13 vs.-0.06, p=0.3) when compared to CG. All IP exhibited similar improvements across rounds, regardless of feedback (p>0.05). In contrast, UP in the FG improved more than CG counterparts in NH (p=0.015), suggesting that feedback was particularly valuable for UP. CONCLUSIONS: AI-based feedback facilitates the acquisition of robotic technical skills by surgical trainees, especially UPs. Future research will extend AI-based feedback to additional suturing skills, surgical tasks, and experience groups. Source of Funding: Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under Award Number R01CA251579. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health © 2023 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 209Issue Supplement 4April 2023Page: e832 Advertisement Copyright & Permissions© 2023 by American Urological Association Education and Research, Inc.MetricsAuthor Information Jasper A. Laca More articles by this author Dani Kiyasseh More articles by this author Rafal Kocielnik More articles by this author Taseen F. Haque More articles by this author Runzhuo Ma More articles by this author Anima Anandkumar More articles by this author Andrew J. Hung More articles by this author Expand All Advertisement PDF downloadLoading ...