Video Face Tracking and Recognition with Skin Region Extraction and Deformable Template Matching

The authors describe a face tracking and recognition system for video indexing that handles variable face poses left-right and up-down and deformations due to speech and facial expressions. The system is based on deformable template matching, and employs person-specific templates at near-frontal poses for recognition, and novel person-independent templates at multiple poses on the view-sphere for tracking. Relative to an earlier version that used multiple person-specific templates at multiple left-right poses, the new system speeds up processing by i restricting attention to skin-color regions; ii performing recognition using the person-specific templates at near-frontal poses only; and iii tracking at non-frontal poses using the novel person-independent templates. Registration is also simplified since multiple views of each target individual are no longer required, but at the cost of a loss of recognition functionality at poses far from frontal the system instead "remembers" the identity of each individual from near-frontal matches and tracks between them.