Modeling Exploration/Exploitation Decisions through Mobile Sensing for Understanding Mechanisms of Addiction

Addiction is a brain disease manifested by the loss of control over drugs or behaviors, despite negative consequences. Although addiction research has been conducted for decades in psychiatry and neuroscience, a comprehensive understanding of the mechanisms underlying addiction has not yet been achieved. Recent studies in neuroscience [1] have sought to bring light upon this issue by measuring exploration/exploitation decisions in sequential choice tasks, requiring balancing the need to exploit known options and to explore new ones. These studies show a relationship between addiction and exploration/exploitation decisions. For example, people addicted to substances (e.g. alcohol or methamphetamine) or behaviors (e.g. gambling) have tendencies to explore less, which implies they have difficulties 'seeing the big picture". There is a small yet growing literature modeling explore/exploit decisions of addicted people through inverse reinforcement learning (IRL) [4]. In this previous work, the models made by decision history of addicted people have higher learning weights and probability to exploit than those of normal people, which means that addicted people are more sensitive to their most recent activity. However, existing methods to measure exploration/exploitation decisions are lab-based game experiments such as n-armed bandit [3] or clock task [6], which are high cost, time-consuming and not scalable. Therefore, they are not suitable for modeling through IRL, which requires large behavioral trajectories. In this work, we argue for the first time that mobile sensing is a more cost-efficient and scalable alternative to the lab-based game experiments for understanding and modeling the mechanisms of addiction (Figure 1).