Facial feature extraction with a depth AAM algorithm.

Facial feature extraction in video sequences takes an important role in face recognition, expression profile analysis, and human computer interaction. Traditional AAM (Active Appearance Model) methods for facial features localization always concentrate on fitting efficiency with few concrete analysis of the characteristic of the initial position and model instance, thus the location accuracy and speed are both not ideal. The main idea of our method is to use the face detection algorithm with a Kinect camera to accurately locate human head and estimate head pose. A depth AAM algorithm is developed to locate the detailed facial features. The head position and pose are used to initialize the AAM global shape transformation which guarantees the model fitting to the correct location. The depth AAM algorithm takes four channels-R, G, B, D into our consideration which combines the colors and the depth of input images. To locate facial feature robustly and accurately, the weights of RGB information and D information in global energy function are adjusted automatically. We also use the image pyramid algorithm and the inverse compositional algorithm to speed up the iteration. Experimental results show that our depth AAM algorithm can effectively and accurately locale facial features from video objects in conditions of complex backgrounds and various poses. © 2012 IEEE.