Validity of tremor analysis using smartphone-compatible computer vision frameworks – a comparative study

Abstract Computer vision (CV)-based approaches hold a promising potential for the classification and quantitative assessment of movement disorders. To fully utilize these, pipelines have to be validated against established clinical and electrophysiological gold standards. This study examines the validity of the Mediapipe (by Google) and Vision (by Apple) smartphone-compatible hand detection frameworks for tremor analysis. Both frameworks were tested in virtual experiments with simulated tremulous hands to determine the optimal camera position for hand tremor assessment and the minimal detectable tremor amplitude and frequency. Then, both frameworks were compared with optical motion capture (OMC), accelerometry and clinical ratings in 20 tremor patients. Both CV frameworks measured tremor peak frequency accurately. Significant correlations were found between the CV-assessed tremor amplitudes and the Essential Tremor Rating Assessment Scale ratings (TETRAS). However, the accuracy of amplitude estimation compared to OMC as ground truth was insufficient for a clinical application. In conclusion CV-based tremor is an accurate and simple clinical assessment tool to determine tremor frequency. Further enhancements are necessary regarding the amplitude estimation.