Chain of Logic: Rule-Based Reasoning with Large Language Models
CoRR(2024)
摘要
Rule-based reasoning, a fundamental type of legal reasoning, enables us to
draw conclusions by accurately applying a rule to a set of facts. We explore
causal language models as rule-based reasoners, specifically with respect to
compositional rules - rules consisting of multiple elements which form a
complex logical expression. Reasoning about compositional rules is challenging
because it requires multiple reasoning steps, and attending to the logical
relationships between elements. We introduce a new prompting method, Chain of
Logic, which elicits rule-based reasoning through decomposition (solving
elements as independent threads of logic), and recomposition (recombining these
sub-answers to resolve the underlying logical expression). This method was
inspired by the IRAC (Issue, Rule, Application, Conclusion) framework, a
sequential reasoning approach used by lawyers. We evaluate chain of logic
across eight rule-based reasoning tasks involving three distinct compositional
rules from the LegalBench benchmark and demonstrate it consistently outperforms
other prompting methods, including chain of thought and self-ask, using
open-source and commercial language models.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要