Evaluating Agents using Social Choice Theory
CoRR(2023)
摘要
We argue that many general evaluation problems can be viewed through the lens
of voting theory. Each task is interpreted as a separate voter, which requires
only ordinal rankings or pairwise comparisons of agents to produce an overall
evaluation. By viewing the aggregator as a social welfare function, we are able
to leverage centuries of research in social choice theory to derive principled
evaluation frameworks with axiomatic foundations. These evaluations are
interpretable and flexible, while avoiding many of the problems currently
facing cross-task evaluation. We apply this Voting-as-Evaluation (VasE)
framework across multiple settings, including reinforcement learning, large
language models, and humans. In practice, we observe that VasE can be more
robust than popular evaluation frameworks (Elo and Nash averaging), discovers
properties in the evaluation data not evident from scores alone, and can
predict outcomes better than Elo in a complex seven-player game. We identify
one particular approach, maximal lotteries, that satisfies important
consistency properties relevant to evaluation, is computationally efficient
(polynomial in the size of the evaluation data), and identifies game-theoretic
cycles
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要