Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompaniment
arxiv(2024)
摘要
We introduce a novel task within the field of 3D dance generation, termed
dance accompaniment, which necessitates the generation of responsive movements
from a dance partner, the "follower", synchronized with the lead dancer's
movements and the underlying musical rhythm. Unlike existing solo or group
dance generation tasks, a duet dance scenario entails a heightened degree of
interaction between the two participants, requiring delicate coordination in
both pose and position. To support this task, we first build a large-scale and
diverse duet interactive dance dataset, DD100, by recording about 117 minutes
of professional dancers' performances. To address the challenges inherent in
this task, we propose a GPT-based model, Duolando, which autoregressively
predicts the subsequent tokenized motion conditioned on the coordinated
information of the music, the leader's and the follower's movements. To further
enhance the GPT's capabilities of generating stable results on unseen
conditions (music and leader motions), we devise an off-policy reinforcement
learning strategy that allows the model to explore viable trajectories from
out-of-distribution samplings, guided by human-defined rewards. Based on the
collected dataset and proposed method, we establish a benchmark with several
carefully designed metrics.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要