Implementation of an Extended Kalman Filter for robust real-time motion capture using IR cameras and optical markers

The technology for tracking reflective markers in 3D using IR cameras is currently mature and widely available, which makes optical motion capture the most popular approach for analyzing human motion. The method consists of recording the trajectories of several markers attached to anatomical landmarks, and then using these trajectories to reconstruct the motion of a rigid multibody model that represents the underlying skeletal system. This method has multiple sources of error, due to the inaccuracy inherent to biomechanical systems. Unless CT scan or similar data is available, the exact geometry of the skeletal system is unknown. In addition, the ideal joints commonly used in multibody models do not always represent the actual kinematics with accuracy. Furthermore, the marker trajectories are perturbed by noise from the capture system and, more importantly, they undergo significant displacements relative to the skeleton, known as skin motion artifact. Figure 1: Marker configuration and multibody model. Obtaining the best estimate of the skeletal motion requires the solution of two problems, which are commonly tackled using optimization techniques: first, the geometry of the multibody model must be fit to the actual skeleton, and secondly, its motion must be adjusted to track the recorded marker trajectories as closely as possible. There exist several methods in the literature for addressing these tasks [1-3], although in general they carry out the motion reconstruction as a post process, so that real-time applications such as Virtual Reality are not possible. In order to obtain real-time motion reconstruction, an Extended Kalman Filter is proposed in this work. The use of a Kalman filter makes the algorithm quite robust to brief marker occlusions, since the system state can be propagated in time even when data from some sensors is missing. Most published Kalman filters applied to motion capture are designed for inertial sensors, although there exist some implementations based on optical markers [4, 5]. However, the latter do not address real-time capture of a full-body human model. The multibody model used in this work, depicted in Fig. 1, is formed by 18 bodies: pelvis, trunk, neck, head, arms, forearms, hands, thighs, calves, hindfeet and forefeet. All bodies are connected through spherical joints, although other joint types can be easily introduced. The filter is based on a Discrete Wiener Process Acceleration model (DWPA) [6], and comprises 189 state variables: position, velocity and acceleration of the pelvis origin (9 states), 18 quaternions representing the orientation of all body segments (72 states), and their corresponding angular velocities and accelerations in local coordinates (108 states). The plant model integrates the quaternions from instant n to n + 1 using the following expression: qi n+1 = qi n + 1 2 qi ⨂(?̅?i Δt + 1 2 ?̅?i Δt2) (1) with a time step Δt of 10 milliseconds, imposed by the motion capture system. The quaternion representing the orientation of body i is noted as qi, and ?̅?i and ?̅?i are the corresponding angular velocity and acceleration. The symbol ⨂ represents the Hamilton’s quaternion product (?̅?i and ?̅?i are treated as quaternions with null scalar part). Since the components of the quaternions are dependent, a normalization must be performed after the Kalman filter correction. In the proposed filter, 36 optical markers act as position sensors. Each marker is attached to a specific body, with fixed local coordinates. These marker coordinates, along with the model geometry, are estimated at a preprocessing stage, where a reference multibody model is scaled to fit the captured markers, by using per-body orthogonal scale factors as design parameters.