Designing and Developing a Novel Deep Computer Vision Platform for Intraoperative Prediction and Analytics

INTRODUCTION: Robust artificial intelligence (AI)-based surgical video analysis platforms could lead to novel insights for intraoperative guidance. METHODS: Microvascular decompression surgeries were recorded using a Storz endoscope. A dataset of 2611 frames from 4 operative videos were segmented to create ground truth images. A sparse labeling paradigm was used, and training data comprised only 3% of the total video frames. Surgical anatomy including the brain stem, cerebellum, cranial nerves, vascular structures and surgical instruments were annotated. We developed a custom deep learning framework built on top of a state-of-the-art instance segmentation algorithm SOLOv2, the baseline. Model was trained for each video and pre-training was transferred across videos. Mean average precision(mAP) was computed within and across videos as an evaluation metric. Lastly, a novel metric to quantify arterial pulsation-induced nerve deformation is introduced, and compared before and after Teflon sponge placement. RESULTS: Our model consistently outperforms the baseline, with an average mAP of 63.75 for within video, and shows feasibility on novel patient video test set with mAP of 40.13 using few-shot learning. For the novel test video, our pulsatility index during compression is 9.5 whereas after decompression is 6.0, indicating successful dampening of artery-nerve pulsation after sponge insertion. CONCLUSIONS: In a sparse labeling paradigm, we design and develop a custom deep computer vision-based instance segmentation architecture to predict and track anatomical structures and surgical objects with high accuracy. We create a novel metric, the “pulsatility index”, which is able to quantify the nerve-artery interface for the first time. Once correlated with outcome measures, real-time AI-based video feed analysis may have the transformative potential to re-define intraoperative standard of care.