Learning Topology Uniformed Face Mesh by Volume Rendering for Multi-view Reconstruction
arxiv(2024)
摘要
Face meshes in consistent topology serve as the foundation for many
face-related applications, such as 3DMM constrained face reconstruction and
expression retargeting. Traditional methods commonly acquire topology uniformed
face meshes by two separate steps: multi-view stereo (MVS) to reconstruct
shapes followed by non-rigid registration to align topology, but struggles with
handling noise and non-lambertian surfaces. Recently neural volume rendering
techniques have been rapidly evolved and shown great advantages in 3D
reconstruction or novel view synthesis. Our goal is to leverage the superiority
of neural volume rendering into multi-view reconstruction of face mesh with
consistent topology. We propose a mesh volume rendering method that enables
directly optimizing mesh geometry while preserving topology, and learning
implicit features to model complex facial appearance from multi-view images.
The key innovation lies in spreading sparse mesh features into the surrounding
space to simulate radiance field required for volume rendering, which
facilitates backpropagation of gradients from images to mesh geometry and
implicit appearance features. Our proposed feature spreading module exhibits
deformation invariance, enabling photorealistic rendering seamlessly after mesh
editing. We conduct experiments on multi-view face image dataset to evaluate
the reconstruction and implement an application for photorealistic rendering of
animated face mesh.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要