Rank dependency of rescaled pruning in recurrent neural networks

Abstract

Throughout development and maturity, neural circuits undergo massive synaptic pruning, yielding highly sparse connectivity while preserving robust population-level computations. These population dynamics are often low-dimensional, allowing task-related computations to be formalized as trajectories within latent subspaces. How such low-dimensional dynamics are preserved amid widespread network sparsification remains unclear. Here, we investigate how different synaptic pruning rules shape low-dimensional dynamics and task performance in recurrent neural networks (RNNs). Moving beyond previous approaches focused on random sparsification of low-rank networks or networks with strictly constrained structures, we systematically evaluate how biologically motivated pruning rules interact with a networks underlying rank. We show that post-pruning dynamics and task performance depend critically on the networks initial rank due to distinct eigenspectral characteristics across rank regimes. Combining mathematical analysis with simulations, we demonstrate that pruning with synaptic rescaling preserves low-dimensional dynamics with minimal distortion in low-rank RNNs, but degrades in the high-rank regime. Our findings suggest that low-rank structure, combined with homeostatic synaptic rescaling, is essential for maintaining stable, low-dimensional dynamics in sparse networks. Author summaryIn biological neural networks, population activity is fundamentally shaped by synaptic connection strengths. During development, many of these connections are eliminated through synaptic pruning--a process thought to minimize metabolic costs and optimize computational efficiency by promoting network sparsity. However, how networks preserve their core functionalities after losing a vast numbers of connections remains an open question. Here, we combine mathematical analysis and simulations to examine how various pruning strategies affect network dynamics and computation in both low- and high-rank recurrent neural networks (RNNs). We show that uncompensated connection removal drastically alters network activity. However, implementing a homeostatic rescaling that strengthens remaining connections preserves original dynamics exclusively in low-rank networks, where connectivity is governed by a few dominant structural patterns. In contrast, pruning with homeostatic rescaling fails to maintain dynamics in high-rank networks. Our findings suggest that low-rank connectivity coupled with network homeostasis is crucial for maintaining brain function throughout significant developmental pruning.