The cost of behavioral flexibility: reversal learning driven by a spiking neural network

Abstract

To survive in a changing world, animals often need to suppress an obsolete behavior and acquire a new one. This process is known as reversal learning (RL). The neural mechanisms underlying RL in spatial navigation have received limited attention and it remains unclear what neural mechanisms maintain behavioral flexibility. We extended an existing closed-loop simulator of spatial navigation and learning, based on spiking neural networks [8]. The activity of place cells and boundary cells were fed as inputs to action selection neurons, which drove the movement of the agent. When the agent reached the goal, behavior was reinforced with spike-timing-dependent plasticity (STDP) coupled with an eligibility trace which marks synaptic connections for future reward-based updates. The modeled RL task had an ABA design, where the goal was switched between two locations A and B every 10 trials. Agents using symmetric STDP excel initially on finding target A, but fail to find target B after the goal switch, persevering on target A. Using asymmetric STDP, using many small place fields, and injecting short noise pulses to action selection neurons were effective in driving spatial exploration in the absence of rewards, which ultimately led to finding target B. However, this flexibility came at the price of slower learning and lower performance. Our work shows three examples of neural mechanisms that achieve flexibility at the behavioral level, each with different characteristic costs.