Taming the chaos gently: a Predictive Alignment learning rule in recurrent neural networks

Abstract

Recurrent neural circuits often face inherent complexities in learning and generating their desired outputs, especially when they initially exhibit chaotic spontaneous activity. While the celebrated FORCE learning rule can train chaotic recurrent networks to produce coherent patterns by suppressing chaos, it requires non-local plasticity rules and extremely quick plasticity, raising the question of how synapses adapt on local, biologically plausible timescales to handle potential chaotic dynamics. We propose a novel framework called "Predictive Alignment", which tames the chaotic recurrent dynamics to generate a variety of patterned activities via a biologically plausible plasticity rule. Unlike most recurrent learning rules, predictive alignment does not aim to directly minimize output error to train recurrent connections, but rather it tries to efficiently suppress chaos by aligning recurrent prediction with chaotic activity. We show that the proposed learning rule can perform supervised learning of multiple target signals, including complex low-dimensional attractors, delay matching tasks that require short-term temporal memory, and finally even dynamic movie clips with high-dimensional pixels. Our findings shed light on how predictions in recurrent circuits can support learning.