Web-based multimedia hand-therapy framework for measuring forward and inverse kinematic data

Recognizing rotational and angular hand movements using a non-invasive way is a challenging task. In this paper, we present a web-based multimedia hand-Therapy framework that can dynamically provide therapy services to a patient having Hemiplegia disability. Using off the shelf 3D motion control sensor called LEAP, the framework can detect, recognize and track different high-level therapeutic movements, both rotational as well as angular, originated from different joints of a Hemiplegic patient’s hand and deduce live kinematic data from these movements. The obtained kinematic data consists of a wide span of hand joint and therapy parameters that is assumed to help medical professionals in their clinical decision making. The multimedia environment uses forward kinematics to receive the live sensory data from the 3D motion tracking sensors and uses inverse kinematics to analyze the sensory data streams in real-time. A subject performs a rehabilitation therapy prescribed by the physician and using both forward and inverse kinematics, the system validates the angular and rotational motion of the joints with respect to the correct therapeutic posture and provides live feedback to the subject. The proposed method is non-invasive as the patient does not need to wear any external devices in the hand and hence can perform therapy exercises even at home. The system uses serious game interfaces to provide immersive and engaging perspective to the therapy domain. To the best of our knowledge, this study is the first of its kind to propose a non-invasive way of tracking live forward and inverse kinematic data in the context of hand therapy. We share the implementation details and our initial test results. Finally, our experiment shows that LEAP has the potential to be used for obtaining kinematic data from primitive and complex hand therapies.