Modeling the Role of the Striatum in Non-Stationary Bandit Tasks

Abstract

Decision making tasks in changing environments with probabilistic reward schemes present various challenges to the agents performing the task. These agents must use the experience gained in the past trials to characterize the environment which guides their actions. We present two models to predict an agents behavior in these tasks - a theoretical model which defines a Bayes optimal solution to the problem under realistic task conditions. The second is a computational model of the basal ganglia which presents a neural mechanism to solve the same. Both the models are shown to reproduce results in behavioral experiments and are compared to each other. This comparison allows us to characterize the theoretical model as a bound on the neural model and the neural model as a biologically plausible implementation of the theoretical model. Furthermore, we predict the performance of the agents in various stochastic regimes which could be tested in future studies.