Surgment: Segmentation-enabled Semantic Search and Creation of Visual Question and Feedback to Support Video-Based Surgery Learning
CoRR(2024)
摘要
Videos are prominent learning materials to prepare surgical trainees before
they enter the operating room (OR). In this work, we explore techniques to
enrich the video-based surgery learning experience. We propose Surgment, a
system that helps expert surgeons create exercises with feedback based on
surgery recordings. Surgment is powered by a few-shot-learning-based pipeline
(SegGPT+SAM) to segment surgery scenes, achieving an accuracy of 92%. The
segmentation pipeline enables functionalities to create visual questions and
feedback desired by surgeons from a formative study. Surgment enables surgeons
to 1) retrieve frames of interest through sketches, and 2) design exercises
that target specific anatomical components and offer visual feedback. In an
evaluation study with 11 surgeons, participants applauded the search-by-sketch
approach for identifying frames of interest and found the resulting image-based
questions and feedback to be of high educational value.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要