Reward predictability shapes decision making states and exploitative control in marmosets

Abstract

Animals navigating dynamic environments must transition between behavioral states dominated by exploitation of known rewards or exploration of alternatives. Understanding this flexibility requires characterizing not only choices and outcomes but also how reward information updates strategy. Common marmosets offer a unique combination of genetic tractability and frontal cortical complexity, yet scalable behavioral platforms for studying flexible decision making in this species remain limited. Here, we developed an automated home-cage touchscreen platform that enables voluntary behavioral testing across extended periods without water restriction. We trained marmosets on a series of reversal learning tasks of increasing complexity, culminating in a fully uncued probabilistic two-armed bandit task, and applied reinforcement learning models to examine how reward predictability shapes adaptive choice in marmosets. A Q-learning model with choice perseveration captured trial-by-trial dynamics, with choice variability consistent with stochastic sampling from the fitted policy. A trial-level behavioral state classifier identified distinct and persistent modes of exploitation and exploration that differed in their sensitivity to reward history and their transition structure. Comparing behavioral patterns within probabilistic and deterministic reward contingencies revealed that reward predictability reorganized state occupancy and sharpened error correction within predominately exploitative states, with animals responding to single negative outcomes more rapidly under deterministic feedback. These findings establish both a behavioral and computational framework to dissect the computations and circuits underlying adaptive decision making in marmosets.