Degree-of-Freedom Matters: Inferring Dynamics from Point Trajectories
CVPR 2024(2024)
摘要
Understanding the dynamics of generic 3D scenes is fundamentally challenging
in computer vision, essential in enhancing applications related to scene
reconstruction, motion tracking, and avatar creation. In this work, we address
the task as the problem of inferring dense, long-range motion of 3D points. By
observing a set of point trajectories, we aim to learn an implicit motion field
parameterized by a neural network to predict the movement of novel points
within the same domain, without relying on any data-driven or scene-specific
priors. To achieve this, our approach builds upon the recently introduced
dynamic point field model that learns smooth deformation fields between the
canonical frame and individual observation frames. However, temporal
consistency between consecutive frames is neglected, and the number of required
parameters increases linearly with the sequence length due to per-frame
modeling. To address these shortcomings, we exploit the intrinsic
regularization provided by SIREN, and modify the input layer to produce a
spatiotemporally smooth motion field. Additionally, we analyze the motion field
Jacobian matrix, and discover that the motion degrees of freedom (DOFs) in an
infinitesimal area around a point and the network hidden variables have
different behaviors to affect the model's representational power. This enables
us to improve the model representation capability while retaining the model
compactness. Furthermore, to reduce the risk of overfitting, we introduce a
regularization term based on the assumption of piece-wise motion smoothness.
Our experiments assess the model's performance in predicting unseen point
trajectories and its application in temporal mesh alignment with guidance. The
results demonstrate its superiority and effectiveness. The code and data for
the project are publicly available:
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要