Decision Theoretic Foundations for Experiments Evaluating Human Decisions
CoRR(2024)
摘要
Decision-making with information displays is a key focus of research in areas
like explainable AI, human-AI teaming, and data visualization. However, what
constitutes a decision problem, and what is required for an experiment to be
capable of concluding that human decisions are flawed in some way, remain open
to speculation. We present a widely applicable definition of a decision problem
synthesized from statistical decision theory and information economics. We
argue that to attribute loss in human performance to forms of bias, an
experiment must provide participants with the information that a rational agent
would need to identify the normative decision. We evaluate the extent to which
recent evaluations of decision-making from the literature on AI-assisted
decisions achieve this criteria. We find that only 6 (17%) of 35 studies that
claim to identify biased behavior present participants with sufficient
information to characterize their behavior as deviating from good
decision-making. We motivate the value of studying well-defined decision
problems by describing a characterization of performance losses they allow us
to conceive. In contrast, the ambiguities of a poorly communicated decision
problem preclude normative interpretation. We conclude with recommendations for
practice.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要