Graded striatal learning factors enable switches between goal-directed and habitual modes, by reassigning behavior control to the fastest-computed representation that predicts reward

Abstract

Different compartments of striatum mediate distinctive behavior-control modes, notably goal-directed versus habitual behavior. Normally, animals move back and forth between these modes as they adapt to changing contingencies of reward. However, this ability is compromised when dopaminergic drugs are used as reinforcers. These facts suggest that a set of biological variables, which make striatal decision making both highly plastic and uniquely sensitive to dopamine, contribute both to normal switches among modes and to the susceptibility for excessive habit formation when dopaminergic drugs serve as rewards. Indeed, data have revealed an impressive number of plasticity- and dopamine-related neural factors that vary systematically (with either increasing or decreasing gradients) across the rostral-ventral-medial to caudal-dorsal-lateral axis within striatum, the same axis implicated in switches among behavioral modes. Computer simulations reported here show how a dopamine-dependent parallel learning algorithm, if applied within modeled cortico-striatal circuits with parameters that reflect these striatal gradients, can explain normal mode switching, both into the habitual mode and returns to goal-directed mode, while also exhibiting a susceptibility to excessive habit formation when a dopaminergic drug serves as reward. With the same parameters, the model also directly illuminates: why interval and probabilistic reinforcement schedules are more habit forming than fixed-ratio schedules; why extinction learning is not (and should not be) a mirror image of acquisition learning; and why striatal decisions guided by reward-guided learning typically exhibit a highly sensitive tradeoff between speed and accuracy.