Real-time face view correction for front-facing cameras

Face views are particularly important in person-to-person communication.Differenes between the camera location and the face orientation can result in undesirable facial appearances of the participants during video conferencing.This phenomenon is par-ticularly noticeable when using devices where the front-facing camera is placed in unconventional locations such as below the display or within the keyboard.In this paper,we take a video stream from a single RGB camera as input,and generate a video stream that emulates the view from a virtual camera at a designated location.The most challenging issue in this problem is that the corrected view often needs out-of-plane head rotations.To address this challenge,we reconstruct the 3D face shape and re-render it into synthesized frames according to the virtual camera location.To output the corrected video stream with natural appearance in real time,we propose several novel techniques including accurate eyebrow reconstruction,high-quality blending between the corrected face image and background,and template-based 3D reconstruction of glasses.Our system works well for different lighting conditions and skin tones,and can handle users wearing glasses.Extensive experiments and user studies demonstrate that our method provides high-quality results.