HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Models
arxiv(2023)
摘要
We address the problem of generating realistic 3D human-object interactions
(HOIs) driven by textual prompts. To this end, we take a modular design and
decompose the complex task into simpler sub-tasks. We first develop a
dual-branch diffusion model (HOI-DM) to generate both human and object motions
conditioned on the input text, and encourage coherent motions by a
cross-attention communication module between the human and object motion
generation branches. We also develop an affordance prediction diffusion model
(APDM) to predict the contacting area between the human and object during the
interactions driven by the textual prompt. The APDM is independent of the
results by the HOI-DM and thus can correct potential errors by the latter.
Moreover, it stochastically generates the contacting points to diversify the
generated motions. Finally, we incorporate the estimated contacting points into
the classifier-guidance to achieve accurate and close contact between humans
and objects. To train and evaluate our approach, we annotate BEHAVE dataset
with text descriptions. Experimental results on BEHAVE and OMOMO demonstrate
that our approach produces realistic HOIs with various interactions and
different types of objects.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要