HGOT: Hierarchical Graph of Thoughts for Retrieval-Augmented In-Context Learning in Factuality Evaluation
CoRR(2024)
摘要
With the widespread adoption of large language models (LLMs) in numerous
applications, the challenge of factuality and the propensity for hallucinations
raises significant concerns. To address this issue, particularly in
retrieval-augmented in-context learning, we introduce the hierarchical graph of
thoughts (HGOT), a structured, multi-layered graph approach designed to enhance
the retrieval of pertinent passages during in-context learning. The framework
utilizes the emergent planning capabilities of LLMs, employing the
divide-and-conquer strategy to break down complex queries into manageable
sub-queries. It refines self-consistency majority voting for answer selection,
which incorporates the recently proposed citation recall and precision metrics
to assess the quality of thoughts, linking an answer's credibility
intrinsically to the thought's quality. This methodology introduces a weighted
system in majority voting, prioritizing answers based on the citation quality
of their thoughts. Additionally, we propose a scoring mechanism for evaluating
retrieved passages, considering factors such as citation frequency and quality,
self-consistency confidence, and the retrieval module's ranking. Experiments
reveal that HGOT outperforms other retrieval-augmented in-context learning
methods, including Demonstrate-Search-Predict (DSP), ReAct, Self-Ask, and
Retrieve-then-Read on different datasets by as much as 7%, demonstrating its
efficacy in enhancing the factuality of LLMs.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要