SurMo: Surface-based 4D Motion Modeling for Dynamic Human Rendering
CVPR 2024(2024)
摘要
Dynamic human rendering from video sequences has achieved remarkable progress
by formulating the rendering as a mapping from static poses to human images.
However, existing methods focus on the human appearance reconstruction of every
single frame while the temporal motion relations are not fully explored. In
this paper, we propose a new 4D motion modeling paradigm, SurMo, that jointly
models the temporal dynamics and human appearances in a unified framework with
three key designs: 1) Surface-based motion encoding that models 4D human
motions with an efficient compact surface-based triplane. It encodes both
spatial and temporal motion relations on the dense surface manifold of a
statistical body template, which inherits body topology priors for
generalizable novel view synthesis with sparse training observations. 2)
Physical motion decoding that is designed to encourage physical motion learning
by decoding the motion triplane features at timestep t to predict both spatial
derivatives and temporal derivatives at the next timestep t+1 in the training
stage. 3) 4D appearance decoding that renders the motion triplanes into images
by an efficient volumetric surface-conditioned renderer that focuses on the
rendering of body surfaces with motion learning conditioning. Extensive
experiments validate the state-of-the-art performance of our new paradigm and
illustrate the expressiveness of surface-based motion triplanes for rendering
high-fidelity view-consistent humans with fast motions and even
motion-dependent shadows. Our project page is at:
https://taohuumd.github.io/projects/SurMo/
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要