MM-LLMs: Recent Advances in MultiModal Large Language Models
CoRR(2024)
摘要
In the past year, MultiModal Large Language Models (MM-LLMs) have undergone
substantial advancements, augmenting off-the-shelf LLMs to support MM inputs or
outputs via cost-effective training strategies. The resulting models not only
preserve the inherent reasoning and decision-making capabilities of LLMs but
also empower a diverse range of MM tasks. In this paper, we provide a
comprehensive survey aimed at facilitating further research of MM-LLMs.
Specifically, we first outline general design formulations for model
architecture and training pipeline. Subsequently, we provide brief
introductions of 26 existing MM-LLMs, each characterized by its specific
formulations. Additionally, we review the performance of MM-LLMs on mainstream
benchmarks and summarize key training recipes to enhance the potency of
MM-LLMs. Lastly, we explore promising directions for MM-LLMs while concurrently
maintaining a real-time tracking website for the latest developments in the
field. We hope that this survey contributes to the ongoing advancement of the
MM-LLMs domain.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要