Automatic landmark detection in distal femur from suraical video footage

Total knee arthroplasty (TKA) is a surgical procedure that consists in replacing the entire knee joint by artificial knee implants. Computer-based navigation systems have been investigated and developed to improve the outcome of TKA procedures. These systems support the surgeon in planning the most adequate position for the implants and assist during the procedure in effectively following the defined surgical plan. This work tackles image-free TKA navigation, which requires the acquisition of particular anatomical landmarks intra-operatively. The accurate localisation of these anatomical landmarks in the distal femur bone is essential for the success of the surgery. However, the landmark identification process is often conducted manually, which is time-consuming, lacks accuracy and has high variability. This work presents an algorithm for automatic detection and localisation of bone landmarks from RGB images acquired during the surgery. It proposes new geometric algorithms for computing the anatomical landmarks in 3D models of the femur, which are used for annotating the images of the surgical footage. The annotated images are then used for training a deep learning-based model, which is able to infer anatomical landmarks from a single RGB image. The experimental results using real surgery data show encouraging performance, being able to generalise for unseen data and presenting reliable predictions.