An Expert is Worth One Token: Synergizing Multiple Expert LLMs as Generalist via Expert Token Routing
arxiv(2024)
摘要
We present Expert-Token-Routing, a unified generalist framework that
facilitates seamless integration of multiple expert LLMs. Our framework
represents expert LLMs as special expert tokens within the vocabulary of a meta
LLM. The meta LLM can route to an expert LLM like generating new tokens.
Expert-Token-Routing not only supports learning the implicit expertise of
expert LLMs from existing instruction dataset but also allows for dynamic
extension of new expert LLMs in a plug-and-play manner. It also conceals the
detailed collaboration process from the user's perspective, facilitating
interaction as though it were a singular LLM. Our framework outperforms various
existing multi-LLM collaboration paradigms across benchmarks that incorporate
six diverse expert domains, demonstrating effectiveness and robustness in
building generalist LLM system via synergizing multiple expert LLMs.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要