FT2TF: First-Person Statement Text-To-Talking Face Generation
CoRR(2023)
摘要
Talking face generation has gained immense popularity in the computer vision
community, with various applications including AR/VR, teleconferencing, digital
assistants, and avatars. Traditional methods are mainly audio-driven ones which
have to deal with the inevitable resource-intensive nature of audio storage and
processing. To address such a challenge, we propose FT2TF - First-Person
Statement Text-To-Talking Face Generation, a novel one-stage end-to-end
pipeline for talking face generation driven by first-person statement text.
Moreover, FT2TF implements accurate manipulation of the facial expressions by
altering the corresponding input text. Different from previous work, our model
only leverages visual and textual information without any other sources (e.g.
audio/landmark/pose) during inference. Extensive experiments are conducted on
LRS2 and LRS3 datasets, and results on multi-dimensional evaluation metrics are
reported. Both quantitative and qualitative results showcase that FT2TF
outperforms existing relevant methods and reaches the state-of-the-art. This
achievement highlights our model capability to bridge first-person statements
and dynamic face generation, providing insightful guidance for future work.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要