Spatiotemporal analysis of speckle dynamics to track invisible needle in ultrasound sequences using Convolutional Neural Networks

Objective Accurate needle placement to the target point is critical for ultrasound interventions like biopsies and epidural injections. However, aligning the needle to the thin plane of the transducer is a challenging issue as it leads to the decay of visibility by the naked eye. Therefore, we have developed a CNN-based framework to track the needle using the spatiotemporal features of speckle dynamics. Methods There are three key techniques to optimize the network for our application. First, we proposed a motion field estimation network (RMF) to extract spatiotemporal features from the stack of consecutive frames. We also designed an efficient network based on the state-of-the-art Yolo framework (nYolo). Lastly, the Assisted Excitation (AE) module was added at the neck of the network to handle imbalance problem. Results Ten freehand ultrasound sequences are collected by inserting an injection needle steeply into the Ultrasound Compatible Lumbar Epidural Simulator and Femoral Vascular Access Ezono test phantoms. We divided the dataset into two sub-categories. In the second category, in which the situation is more challenging and the needle is totally invisible statically, the angle and tip localization error were 2.43±1.14° and 2.3±1.76 mm using Yolov3+RMF+AE and 2.08±1.18° and 2.12±1.43 mm using nYolo+RMF+AE. Conclusion and significance The proposed method has the potential to track the needle in a more reliable operation compared to other state-of-the-art methods and can accurately localize it in 2D B-mode US images in real-time, allowing it to be used in in current ultrasound intervention procedures. ### Competing Interest Statement The authors have declared no competing interest.