Dissociation between task structure learning and performance in human model-based reinforcement learning

The multi-step learning paradigm has become the dominant paradigm to investigate the trade-off between model-free reinforcement learning – which only leverages state-action-reward associations – and model-based reinforcement learning – which additionally builds on an explicit representation of state-transitions. Experimentally, while reward values usually have to be learned by trial-and-errors, state-transitions are customarily provided by instructions and extensive training. Accordingly, little is known about the learning strategies that are implemented in the ecological situation in which action-state-transitions are not known ex-ante. To fill this gap, we administered a new version of the two-step tasks, in which action-state-transitions have to be learned in parallel of reward values, to two cohorts of participants tested in the lab (N=30) and online (N=200). While choice patterns in the learning phase showed little sign of model-based learning, participants still accurately retrieved action-state-transitions in post-learning assessments. Together our results reveal an intriguing dissociation between knowledge and performance in human reinforcement learning: while reward values and the action-state-transitions can be concomitantly learned in the two-step paradigm, choices do not seem to benefit from model-based computations.