SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generation
arxiv(2023)
摘要
Our goal is to synthesize 3D human motions given textual inputs describing
simultaneous actions, for example 'waving hand' while 'walking' at the same
time. We refer to generating such simultaneous movements as performing 'spatial
compositions'. In contrast to temporal compositions that seek to transition
from one action to another, spatial compositing requires understanding which
body parts are involved in which action, to be able to move them
simultaneously. Motivated by the observation that the correspondence between
actions and body parts is encoded in powerful language models, we extract this
knowledge by prompting GPT-3 with text such as "what are the body parts
involved in the action ?", while also providing the parts list and
few-shot examples. Given this action-part mapping, we combine body parts from
two motions together and establish the first automated method to spatially
compose two actions. However, training data with compositional actions is
always limited by the combinatorics. Hence, we further create synthetic data
with this approach, and use it to train a new state-of-the-art text-to-motion
generation model, called SINC ("SImultaneous actioN Compositions for 3D human
motions"). In our experiments, that training with such GPT-guided synthetic
data improves spatial composition generation over baselines. Our code is
publicly available at https://sinc.is.tue.mpg.de/.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要