Debatrix: Multi-dimensional Debate Judge with Iterative Chronological Analysis Based on LLM
arxiv(2024)
摘要
How can we construct an automated debate judge to evaluate an extensive,
vibrant, multi-turn debate? This task is challenging, as judging a debate
involves grappling with lengthy texts, intricate argument relationships, and
multi-dimensional assessments. At the same time, current research mainly
focuses on short dialogues, rarely touching upon the evaluation of an entire
debate. In this paper, by leveraging Large Language Models (LLMs), we propose
Debatrix, which makes the analysis and assessment of multi-turn debates more
aligned with majority preferences. Specifically, Debatrix features a vertical,
iterative chronological analysis and a horizontal, multi-dimensional evaluation
collaboration. To align with real-world debate scenarios, we introduced the
PanelBench benchmark, comparing our system's performance to actual debate
outcomes. The findings indicate a notable enhancement over directly using LLMs
for debate evaluation. Source code and benchmark data are available online at
https://github.com/ljcleo/debatrix .
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要