δ-CAUSAL: Exploring Defeasibility in Causal Reasoning
CoRR(2024)
摘要
Defeasibility in causal reasoning implies that the causal relationship
between cause and effect can be strengthened or weakened. Namely, the causal
strength between cause and effect should increase or decrease with the
incorporation of strengthening arguments (supporters) or weakening arguments
(defeaters), respectively. However, existing works ignore defeasibility in
causal reasoning and fail to evaluate existing causal strength metrics in
defeasible settings. In this work, we present δ-CAUSAL, the first
benchmark dataset for studying defeasibility in causal reasoning.
δ-CAUSAL includes around 11K events spanning ten domains, featuring
defeasible causality pairs, i.e., cause-effect pairs accompanied by supporters
and defeaters. We further show current causal strength metrics fail to reflect
the change of causal strength with the incorporation of supporters or defeaters
in δ-CAUSAL. To this end, we propose CESAR (Causal Embedding aSsociation
with Attention Rating), a metric that measures causal strength based on
token-level causal relationships. CESAR achieves a significant 69.7
improvement over existing metrics, increasing from 47.2
the causal strength change brought by supporters and defeaters. We further
demonstrate even Large Language Models (LLMs) like GPT-3.5 still lag 4.5 and
10.7 points behind humans in generating supporters and defeaters, emphasizing
the challenge posed by δ-CAUSAL.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要