Forward-Backward Reasoning in Large Language Models for Mathematical Verification
arxiv(2023)
摘要
Self-Consistency samples diverse reasoning chains with answers and chooses
the final answer by majority voting. It is based on forward reasoning and
cannot further improve performance by sampling more reasoning chains when
saturated. To further boost performance, we introduce backward reasoning to
verify candidate answers. Specifically, for mathematical tasks, we mask a
number in the question and ask the LLM to answer a backward question created by
a simple template, i.e., to predict the masked number when a candidate answer
is provided. Instead of using forward or backward reasoning alone, we propose
FOBAR to combine FOrward and BAckward Reasoning for verification. Extensive
experiments on six standard mathematical data sets and three LLMs show that
FOBAR achieves state-of-the-art performance. In particular, FOBAR outperforms
Self-Consistency, which uses forward reasoning alone, demonstrating that
combining forward and forward reasoning is better. In addition, FOBAR performs
better than existing verification methods, showing the effectiveness of the
simple template used in backward reasoning and the proposed combination.
Extensions to non-mathematical problems are also discussed and validated
empirically.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要