Skilled Bandits: Learning To Choose In A Reactive World

In uncertain environments we must balance our need to gather information with our desire to reap rewards by exploiting current knowledge. Achieving this balance is further complicated in reactive environments where actions produce long-lasting change to the system. In four experiments, we investigate how people learn to make effective decisions from experience in a dynamic multiarmed bandit task. In contrast to the typical exploitation-dependent diminishing rewards found in previous studies, options were framed as skills that developed greater rewards the more they were chosen. In Experiment 1, we provide a proof of concept. and in Experiments 2-4 we explore the boundaries of participants' sensitivity to reactive dynamics. Our results suggest that most individuals can learn effective strategies for coping with these reactive environments. A two-part comparison of several competing psychological models supports several conclusions: (a) a sizable minority of individuals learned that their environment was reactive, (b) evidence suggests several distinct groups of individuals employed unique decision strategies, and (c) testing models with the simulation method reveals qualitative misfits that motivate future theory development.