Analysis of effective speech recognition in A Virtual Operating Room

Recently, we have developed a simulated environment for training surgical team members in judgement, decisionmaking, and technical ability. The Virtual Operating Room (VOR) is a fully immersive virtual environment that augments procedural task simulator training with a simulated OR context. The VOR is modeled on a standard OR and outfitted with both real and virtual equipment and a commercial medical simulator for laparoscopic cholecystectomy. Trainees communicate with virtual teammates using speech recognition software. One challenge with the VOR is developing a system for speech recognition that includes all reasonable surgical utterances while retaining flexibility between differences in user terminology. In the initial configuration, speech recognition rules were rigid and required operators to manually intervene for statements that went unrecognized. In the current configuration, we developed an extended state machine formalism that reduced the number of words needed to be detected in each utterance. The present paper compares previous results with the current version. Surgical residents were recruited to test system efficacy using free speech. Surgeons’ perceptions about speech recognition with the current version were more positive compared to older version: levels of dissatisfaction with the vocal control declined from 20% to zero. Additionally, a third of the surgeons reported feeling as though the simulation was realistic due to the ability to communicate with the system via speech. However, some additional recognition issues were revealed, including lag in system response times, confusion regarding system inquiries and appropriate responses, the need to provide verbal feedback for certain procedural steps, and unexpected user responses. Preliminary results suggest that communication efficacy in the current VOR has improved, but issues such as system response time and speech recognition accuracy still pose a challenge.