IAG: Induction-Augmented Generation Framework for Answering Reasoning Questions.
Conference on Empirical Methods in Natural Language Processing(2023)
摘要
Retrieval-Augmented Generation (RAG), by incorporating external knowledge
with parametric memory of language models, has become the state-of-the-art
architecture for open-domain QA tasks. However, common knowledge bases are
inherently constrained by limited coverage and noisy information, making
retrieval-based approaches inadequate to answer implicit reasoning questions.
In this paper, we propose an Induction-Augmented Generation (IAG) framework
that utilizes inductive knowledge along with the retrieved documents for
implicit reasoning. We leverage large language models (LLMs) for deriving such
knowledge via a novel prompting method based on inductive reasoning patterns.
On top of this, we implement two versions of IAG named IAG-GPT and IAG-Student,
respectively. IAG-GPT directly utilizes the knowledge generated by GPT-3 for
answer prediction, while IAG-Student gets rid of dependencies on GPT service at
inference time by incorporating a student inductor model. The inductor is
firstly trained via knowledge distillation and further optimized by
back-propagating the generator feedback via differentiable beam scores.
Experimental results show that IAG outperforms RAG baselines as well as ChatGPT
on two Open-Domain QA tasks. Notably, our best models have won the first place
in the official leaderboards of CSQA2.0 (since Nov 1, 2022) and StrategyQA
(since Jan 8, 2023).
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要