Development and Evaluation of a Research Framework for Measuring the Reliance on Automation in Situations of Risk and Moral Dilemma

Digitalization is changing the working world of tomorrow and complex decisions are increasingly being made by hybrid teams of people and automated decision support systems. The interaction of user, task, and interface plays a decisive role, but is insufficiently studied, especially in the context of decision making under uncertainty, risk, and moral dilemmas. In this article, we present a web-based research framework for exploring this factor space with empirical user studies. The framework builds on a combination of tools for linking a user model, with the outcome of game-based decision scenarios, and the participants’ evaluations. To validate the framework, we conducted a user study with N = 64 participants and found that the system responded sensitively to factors from each of the three areas user, task, and interface. Consequently, the framework facilitates the systematic investigation of the factor space, the identification and quantification of relevant factors and the description of interactions between the factors. A systematic understanding of these factors is necessary for the development of future joint cognitive systems and Decision Support Systems.