Model-based Tracking of Magnetic Sensor Gloves in Real Time

While mechanical tracking is commonly seen in robot-assisted surgery, contactless gesture control leads to a more intuitive approach. Magnetic localization systems might be able to provide an untethered access to the necessary hand movement data. A model-based system exploiting a priori knowledge to improve the accuracy and robustness can be beneficial in this context. We present a proof of concept, in which the index finger of a digital twin of a hand is tracked with three simulated sensors. Based on the physiological composition, the finger is modeled as a kinematic chain with rotational degrees of freedom between the segments according to the corresponding finger joints. We applied an extended Kalman filter using this description to enhance position and rotation estimates for the sensors on the finger segments. We achieved mean absolute errors < 1 cm for the positions and < 10 for the local rotations with first simulations of a bending motion.