MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusion
CoRR(2023)
摘要
In this work, we propose MagicPose, a diffusion-based model for 2D human pose
and facial expression retargeting. Specifically, given a reference image, we
aim to generate a person's new images by controlling the poses and facial
expressions while keeping the identity unchanged. To this end, we propose a
two-stage training strategy to disentangle human motions and appearance (e.g.,
facial expressions, skin tone and dressing), consisting of (1) the pre-training
of an appearance-control block and (2) learning appearance-disentangled pose
control. Our novel design enables robust appearance control over generated
human images, including body, facial attributes, and even background. By
leveraging the prior knowledge of image diffusion models, MagicPose generalizes
well to unseen human identities and complex poses without the need for
additional fine-tuning. Moreover, the proposed model is easy to use and can be
considered as a plug-in module/extension to Stable Diffusion.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要