Exploring the psychology of LLMs' Moral and Legal Reasoning
arXiv (Cornell University)(2023)
摘要
Large language models (LLMs) exhibit expert-level performance in tasks across
a wide range of different domains. Ethical issues raised by LLMs and the need
to align future versions makes it important to know how state of the art models
reason about moral and legal issues. In this paper, we employ the methods of
experimental psychology to probe into this question. We replicate eight studies
from the experimental literature with instances of Google's Gemini Pro,
Anthropic's Claude 2.1, OpenAI's GPT-4, and Meta's Llama 2 Chat 70b. We find
that alignment with human responses shifts from one experiment to another, and
that models differ amongst themselves as to their overall alignment, with GPT-4
taking a clear lead over all other models we tested. Nonetheless, even when
LLM-generated responses are highly correlated to human responses, there are
still systematic differences, with a tendency for models to exaggerate effects
that are present among humans, in part by reducing variance. This recommends
caution with regards to proposals of replacing human participants with current
state-of-the-art LLMs in psychological research and highlights the need for
further research about the distinctive aspects of machine psychology.
更多查看译文
关键词
legal reasoning,moral,psychology
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要