Learning sculpts orthogonal task manifolds for continual skill learning in recurrent networks

Abstract

Humans and animals can learn and seamlessly perform a vast repertoire of behaviors. However, how neural populations incorporate new skills without disrupting previously learned ones remains poorly understood. This challenge is known as catastrophic forgetting in artificial neural networks and is especially severe in recurrent networks, where computation relies on stable internal dynamics. While current machine learning approaches often rely on explicit weight-protection strategies, such approaches do not directly address this challenge. Here we show that orthogonal task manifolds can emerge in recurrent neural networks from a local predictive, error-driven learning rule. Our model preserves task-specific latent dynamics that remain resilient to interference from subsequent learning. Using low-rank connectivity ablations, we causally isolate these latent dynamics, selectively impairing individual tasks while leaving others intact. We further show that the proposed principle generalizes beyond low-dimensional to high-dimensional naturalistic movie replay, suggesting a scalable mechanism for continual learning. Our results identify a solution to catastrophic forgetting by preserving previously learned dynamics in recurrent connectivity, thereby providing a mechanistic bridge between artificial recurrent networks and biological neural circuits.