MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models
CoRR(2024)
摘要
Multi-agent interactions between Large Language Model (LLM) agents have shown
major improvements on diverse reasoning tasks. However, these involve long
generations from multiple models across several rounds, making them expensive.
Moreover, these multi-agent approaches fail to provide a final, single model
for efficient inference. To address this, we introduce MAGDi, a new method for
structured distillation of the reasoning interactions between multiple LLMs
into smaller LMs. MAGDi teaches smaller models by representing multi-agent
interactions as graphs, augmenting a base student model with a graph encoder,
and distilling knowledge using three objective functions: next-token
prediction, a contrastive loss between correct and incorrect reasoning, and a
graph-based objective to model the interaction structure. Experiments on seven
widely-used commonsense and math reasoning benchmarks show that MAGDi improves
the reasoning capabilities of smaller models, outperforming several methods
that distill from a single teacher and multiple teachers. Moreover, MAGDi also
demonstrates an order of magnitude higher efficiency over its teachers. We
conduct extensive analyses to show that MAGDi (1) enhances the generalizability
to out-of-domain tasks, (2) scales positively with the size and strength of the
base student model, and (3) obtains larger improvements (via our multi-teacher
training) when applying self-consistency - an inference technique that relies
on model diversity.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要