Investigating Agency of LLMs in Human-AI Collaboration Tasks
Conference of the European Chapter of the Association for Computational Linguistics(2023)
摘要
Agency, the capacity to proactively shape events, is central to how humans
interact and collaborate. While LLMs are being developed to simulate human
behavior and serve as human-like agents, little attention has been given to the
Agency that these models should possess in order to proactively manage the
direction of interaction and collaboration. In this paper, we investigate
Agency as a desirable function of LLMs, and how it can be measured and managed.
We build on social-cognitive theory to develop a framework of features through
which Agency is expressed in dialogue - indicating what you intend to do
(Intentionality), motivating your intentions (Motivation), having self-belief
in intentions (Self-Efficacy), and being able to self-adjust (Self-Regulation).
We collect a new dataset of 83 human-human collaborative interior design
conversations containing 908 conversational snippets annotated for Agency
features. Using this dataset, we develop methods for measuring Agency of LLMs.
Automatic and human evaluations show that models that manifest features
associated with high Intentionality, Motivation, Self-Efficacy, and
Self-Regulation are more likely to be perceived as strongly agentive.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要