Using an LLM to Turn Sign Spottings into Spoken Language Sentences
arxiv(2024)
摘要
Sign Language Translation (SLT) is a challenging task that aims to generate
spoken language sentences from sign language videos. In this paper, we
introduce a hybrid SLT approach, Spotter+GPT, that utilizes a sign spotter and
a pretrained large language model to improve SLT performance. Our method builds
upon the strengths of both components. The videos are first processed by the
spotter, which is trained on a linguistic sign language dataset, to identify
individual signs. These spotted signs are then passed to the powerful language
model, which transforms them into coherent and contextually appropriate spoken
language sentences.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要