Simultaneous Estimation of Digit Tip Forces and Hand Postures in a Simulated Real-life Condition with High-density Electromyography and Deep Learning

In myoelectric control, continuous estimation of multiple degrees of freedom has an important role. Most studies have focused on estimating discrete postures or forces of the human hand but for a practical prosthetic system, both should be considered. In daily life activities, hand postures vary for grasping different objects and the amount of force exerted on each fingertip depends on the shape and weight of the object. This study aims to investigate the feasibility of continuous estimation of multiple degrees of freedom. We proposed a reach and grasp framework to study both absolute fingertip forces and hand movement types using deep learning techniques applied to high-density surface electromyography (HD-sEMG). Four daily life grasp types were examined and absolute fingertip forces were simultaneously estimated while grasping various objects, along with the grasp types. We showed that combining a 3-dimensional Convolutional Neural Network (3DCNN) with a Long Short-term Memory (LSTM) can reliably and continuously estimate the digit tip forces and classify different hand postures in human individuals. The mean absolute error (MAE) and Pearson correlation coefficient (PCC) results of the force estimation problem across all fingers and subjects were 0.46 ± 0.23 and 0.90 ± 0.03% respectively and for the classification problem, they were 0.04 ± 0.01 and 0.97 ± 0.02%. The results demonstrated that both absolute digit tip forces and hand postures can be successfully estimated through deep learning and HD-sEMG.