Using Disinhibition versus Direct Control in a Spiking Neural Model of Dopamine-Driven Reinforcement Learning

Abstract

Dopaminergic signalling is central to value learning and decision making. It has been observed that multiple pathways with different patterns of connectivity project to midbrain dopaminergic neurons, some involving direct excitatory projections while others involve disinhibition. However, the respective contributions of these patterns to dopamine control, and their computational and functional advantages remain unclear. In the current work we simulate and evaluate two fully spiking neural models of dopaminergic control, based either solely on disinhibition, or solely on direct inhibitory and excitatory projections. We compare these models in terms of their engineering properties, their resulting spiking profiles, and their ability to successfully acquire representations of expected value in a 3-armed bandit task. We find that both models are able to operate at an asynchronous-irregular firing regime, but that the firing profile of the direct integration model is less resilient to disruption and more sensitive to incoming signals. In addition, the disinhibition model performs better in the learning task. We conclude that while the direct model is more parsimonious, disinhibition-based control remains advantageous in the operational context. Our results have implications for the study of decision-making brain circuits as well as for the design of brain-inspired systems.