Dopaminergic modulation of the exploration/exploitation trade-off in human decision-making

Abstract

A central issue in reinforcement learning and decision-making is whether to exploit knowledge of reward values, or to explore novel options. Although it is widely hypothesized that dopamine neurotransmission plays a key role in regulating this balance, causal evidence for a role of dopamine in human exploration is still lacking. Here, we use a combination of computational modeling, pharmacological intervention and functional magnetic resonance imaging (fMRI) to test for a causal effect of dopamine transmission on the exploration-exploitation trade-off in humans. 31 healthy male subjects performed a restless four-armed bandit task in a within-subjects design under three drug conditions: 150mg of the dopamine precursor L-dopa, 2mg of the D2 receptor antagonist haloperidol, and placebo. In all conditions, choice behavior was best explained by an extension of an established Bayesian learning model accounting for perseveration, uncertainty-based exploration and random exploration. Uncertainty-based exploration was attenuated under L-dopa compared to placebo and haloperidol. There was no evidence for a modulation of prediction error signaling or categorical effects of exploration/exploitation under L-dopa, whereas model-based fMRI revealed that L-dopa attenuated neural representations of overall uncertainty in insula and dorsal anterior cingulate cortex. Our results highlight the computational role of these regions in exploration and suggest that dopamine modulates exploration by modulating how this circuit tracks accumulating uncertainty during decision-making.