Teaching Language Models to Self-Improve through Interactive Demonstrations
arxiv(2023)
摘要
The self-improving ability of large language models (LLMs), enabled by
prompting them to analyze and revise their own outputs, has garnered
significant interest in recent research. However, this ability has been shown
to be absent and difficult to learn for smaller models, thus widening the
performance gap between state-of-the-art LLMs and more cost-effective and
faster ones. To reduce this gap, we introduce TriPosT, a training algorithm
that endows smaller models with such self-improvement ability, and show that
our approach can improve a LLaMA-7b's performance on math and reasoning tasks
by up to 7.13
model to interact with LLMs to collect feedback and improvements on its own
generations. We then replay this experience to train the small model. Our
experiments on four math and reasoning datasets show that the interactive
experience of learning from and correcting its own mistakes is crucial for
small models to improve their performance.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要