Dopaminergic reward signals shape foraging decisions through learned value expectations

Abstract

A fundamental function of the nervous system is to direct animals towards life-sustaining resources and enable adaptive strategies for acquiring them. Foraging, the challenge of securing these resources from the environment, requires making decisions that balance local gains against costs, opening a window on value-based decision making in ecological context. Yet despite the centrality of foraging in life, the algorithms and neural mechanisms underlying foraging decisions are not well understood. We studied mice making stay-or-leave patch foraging decisions for water across poor- and rich-resource environments with dynamic local reward outcomes. We found that mice integrate immediate rewards, local reward history, and reward (i.e. value) expectations to guide foraging decisions, favoring greater time investment in rich-environment patches. Computational models revealed influence of a time-varying bias which mirrored the shape of a learned value function. Consistent with a reinforcement learning mechanism, dopamine in the ventral striatum signaled a temporal-difference reward prediction error, which could be used as a teaching signal to construct value functions, and calibrated optogenetic manipulation of dopamine to create fictive reward signals recapitulated the value-related decision bias. Taken together, our results shed new light on the neurocomputational basis of foraging, revealing a novel role for value expectations in dynamic environments. Further, they establish generalized function of biological reinforcement learning in ecological context, demonstrating a specific role for the canonical dopamine reward circuit in shaping temporally resolved value expectations to guide behavior and cognition. The unification of this framework with foraging opens a new avenue to elucidate principles of how animals learn and dynamically adjust cognitive strategies for real-world ecological problems.