Reconstruction of 3D Faces Based on Monocular Vision

Deep learning is receiving more and more attention in the reconstruction of three-dimensional (3D) faces. However, most of the reported methods use a single 2D image to reconstruct a 3D face, which looks true but is lack of 3D accuracy and personalization. In order reconstruct 3D faces for applications with dimensional accuracy requirement such as medical treatment, we propose a deep learning based multi-view reconstruction method, which aims to provide sufficient 3D constraints to obtain high-quality 3D face models. We propose a multi-view hybrid loss function to calculate the loss at the image level and the perception level. Weakly supervised learning is implemented to train an end-to-end network, and regressed directly to extract the face reconstruction parameters. The multi-view approach not only improve the reconstruction effect of the face in the front, but also from the side, such as the jaw, which is critical in dentistry. We set the region of interest mask to improve the accuracy of reconstruction in specific regions. We have done comparative experiments with several state-of-the-art reconstruction methods, and the results show that the method in this paper can effectively improve the overall 3D face reconstruction accuracy.