K-wire Tracking in 3D Camera Views

K-wires are widely used instruments in orthopedic surgeries. Currently, K-wire insertion is a long, tedious procedure that requires multiple X-rays and mental alignment of patient, wire and X-rays by the doctor. We propose a deep learning based solution to track K-wire in RGB stereo images, which can be then used to detect and track the K-wire in 3D space. The goal is to estimate the orientation of the K-wire in 3D. Due to the shortage of real surgical scene data, we create our own artificial data for training by composing foreground (K-wire) and background separately. We then explore the performance of two different networks for K-wire segmentation, U-Net and HED. Finally, we validate the performance of the two different networks in 2D and 3D space with naturally acquired images and achieve < 1◦ and < 5◦ average error in each respectively.