Policy regularization as a unifying theory of the striatal division of labor in learning

Abstract

Learning novel behaviors requires balancing previously learned actions with the ability to flexibly adapt to changing reward contingencies. This trade-off is well documented in the division of labor between dorsolateral striatum (DLS), which promotes selection of cached, history-dependent actions, and dorsomedial striatum (DMS), which supports flexible learning as reward contingencies change. Here, we propose policy regularization as a general computational principle for understanding this division. Capacity-limited agents face a fundamental trade-off between maximizing reward and minimizing the cost of deviating from a default policy that caches frequently used action transitions. We formalize this trade-off as a KL-regularized reward objective in which a flexible controller (DMS) incurs a cost for diverging from a history-dependent default (DLS). The resulting optimal policy is a combination of a reward-driven action value, continuously updated by DMS, and a default policy conditioned on action history, cached by DLS. In practice, DLS consolidates the action transitions shaped by DMS's reward-driven value learning, so DLS preserves behaviors that were once optimal even when reward contingencies change, producing robust but inflexible action selection and effectively "regularizing" DMS-driven learning. We show that a single model, with a single set of learning and lesion parameters, shared across paradigms, provides a unifying explanation for the functional organization of the striatum, reproducing canonical DLS-DMS dissociations in outcome devaluation, serial spatial reversal, skilled action sequencing, and motor sequence execution tasks.