AnaMoDiff: 2D Analogical Motion Diffusion via Disentangled Denoising
CoRR(2024)
摘要
We present AnaMoDiff, a novel diffusion-based method for 2D motion analogies
that is applied to raw, unannotated videos of articulated characters. Our goal
is to accurately transfer motions from a 2D driving video onto a source
character, with its identity, in terms of appearance and natural movement, well
preserved, even when there may be significant discrepancies between the source
and driving characters in their part proportions and movement speed and styles.
Our diffusion model transfers the input motion via a latent optical flow (LOF)
network operating in a noised latent space, which is spatially aware, efficient
to process compared to the original RGB videos, and artifact-resistant through
the diffusion denoising process even amid dense movements. To accomplish both
motion analogy and identity preservation, we train our denoising model in a
feature-disentangled manner, operating at two noise levels. While
identity-revealing features of the source are learned via conventional noise
injection, motion features are learned from LOF-warped videos by only injecting
noise with large values, with the stipulation that motion properties involving
pose and limbs are encoded by higher-level features. Experiments demonstrate
that our method achieves the best trade-off between motion analogy and identity
preservation.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要