Tremor detection Transformer: An automatic symptom assessment framework based on refined whole-body pose estimation

Essential tremor (ET) is a prevalent neurological disorder that necessitates using objective and non-invasive methods for assessing symptom severity. Traditional visual assessments are often limited by subjectivity, while other wearable sensors or visual markers may result in unnatural movements. This study presents a novel contact-free visual-based pipeline for ET assessment that integrates refined whole-body pose esti-mation with Transformer-based tremor detection to quantify tremor severity at a fine-grained level. The proposed pose estimation method combines the Transformer with HRNet, effectively capturing spatial-temporal complementary information from multiple body parts and enabling highly accurate tremor detection. The Transformer-based tremor detection is well-suited for modeling long-range dependencies and sequential tremor data extracted by the pose estimation model, further improving the performance of our proposed method. Our study collected data from 61 patients with ET, achieving an average accuracy, recall, and F1 score of 95.6%/95.6%, 89.2%/95.0%, and 83.0%/92.4% for classifying ET severity both during rest and postural tasks, respectively. Our proposed method outperforms the temporal convolutional network baseline, increasing F1 scores by 21.17% and 14.22% for rest and postural tasks, respectively. This high level of accuracy makes our method highly useful for clinical applications such as remote monitoring, diagnosis, and treatment evaluation. Our proposed technique has many advantages over traditional ET assessment techniques, including non-invasiveness, contact-free operation, and not requiring any wearable sensors or visual markers. Moreover, our method can be applied to other movement disorders requiring objective measurements of symptom severity. In summary, our contact-free visual-based pipeline for ET assessment represents a significant improvement over traditional ET assessment techniques, and our quantification results demonstrate its potential for use in clinical settings.