Call Me When Necessary: LLMs can Efficiently and Faithfully Reason over Structured Environments
arxiv(2024)
摘要
Large Language Models (LLMs) have shown potential in reasoning over
structured environments, e.g., knowledge graph and table. Such tasks typically
require multi-hop reasoning, i.e., match natural language utterance with
instances in the environment. Previous methods leverage LLMs to incrementally
build a reasoning path, where the LLMs either invoke tools or pick up schemas
by step-by-step interacting with the environment. We propose
Reasoning-Path-Editing (Readi), a novel framework where LLMs can efficiently
and faithfully reason over structured environments. In Readi, LLMs initially
generate a reasoning path given a query, and edit the path only when necessary.
We instantiate the path on structured environments and provide feedback to edit
the path if anything goes wrong. Experimental results on three KGQA datasets
and two TableQA datasets show the effectiveness of Readi, significantly
surpassing all LLM-based methods (by 9.1
on WTQ), comparable with state-of-the-art fine-tuned methods (67
74.7
Our code will be available upon publication.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要