Face de-identification using multi-factor active appearance models

Recent advances in both camera technology as well as supporting computing hardware have made image and video acquisition close to effortless. As a consequence many applications capture image data of people for either immediate inspection or storage and subsequent sharing. These improved recording capabilities, however, ignite concerns about the privacy of people identifiable in the scene. While algorithms have been proposed to de-identify images, currently available methods are lacking in many respects. In this dissertation we develop a general framework for the de-identification of images which subsumes a number of previously introduced approaches. Our goal is to remove as much identifying information as necessary while preserving as much of the original signal as possible. To this extend we propose three privacy protection models and provide multiple algorithms implementing them. We concentrate our work on face images since faces provide strong identity cues that can be extracted from a distance, even for uncooperative and unaware subjects. The appearance of a face depends on a number of factors, including identity, pose, illumination, and facial expression. Algorithms have been developed to factorize images into these underlying factors using linear or bilinear multi-factor models. In this dissertation we propose a unified framework that combines linear, bilinear, and quadratic models. We show how to fit the models to data and discuss ways to avoid overfitting and enforce additional constraints. Through combination of the factorization algorithms with our de-identification framework we are able to preserve more data utility while maintaining privacy protection. We demonstrate this through experiments on the Multi-PIE face database as well as experiments in de-identifying video sequences from a medical face database.