TextBind: Multi-turn Interleaved Multimodal Instruction-following in the Wild
CoRR(2023)
摘要
Large language models with instruction-following abilities have
revolutionized the field of artificial intelligence. These models show
exceptional generalizability to tackle various real-world tasks through their
natural language interfaces. However, their performance heavily relies on
high-quality exemplar data, which is often difficult to obtain. This challenge
is further exacerbated when it comes to multimodal instruction following. We
introduce TextBind, an almost annotation-free framework for empowering larger
language models with the multi-turn interleaved multimodal
instruction-following capabilities. Our approach requires only image-caption
pairs and generates multi-turn multimodal instruction-response conversations
from a language model. To accommodate interleaved image-text inputs and
outputs, we devise MIM, a language model-centric architecture that seamlessly
integrates image encoder and decoder models. We release our dataset, model, and
demo to foster future research in the area of multimodal instruction following.
更多查看译文
关键词
multi-turn,instruction-following
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要