Virtual Pets: Animatable Animal Generation in 3D Scenes
CoRR(2023)
摘要
Toward unlocking the potential of generative models in immersive 4D
experiences, we introduce Virtual Pet, a novel pipeline to model realistic and
diverse motions for target animal species within a 3D environment. To
circumvent the limited availability of 3D motion data aligned with
environmental geometry, we leverage monocular internet videos and extract
deformable NeRF representations for the foreground and static NeRF
representations for the background. For this, we develop a reconstruction
strategy, encompassing species-level shared template learning and per-video
fine-tuning. Utilizing the reconstructed data, we then train a conditional 3D
motion model to learn the trajectory and articulation of foreground animals in
the context of 3D backgrounds. We showcase the efficacy of our pipeline with
comprehensive qualitative and quantitative evaluations using cat videos. We
also demonstrate versatility across unseen cats and indoor environments,
producing temporally coherent 4D outputs for enriched virtual experiences.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要