Random network structure stabilizes neural manifolds

Abstract

Neuronal activity patterns change continuously over days and weeks, a phenomenon known as representational drift. Despite this, the geometric structure of population representations, namely the pairwise similarities between stimulus-evoked activity patterns, remains remarkably stable. How can ongoing changes in activity be consistent with stable representational similarity? We show that this is a generic consequence of random connectivity: in networks with random connectivity, output similarity is a monotonically increasing function of input similarity, independent of the specific connectivity pattern. Drift, whether driven by random synaptic turnover or Hebbian plasticity, merely transitions the network between random instantiations, leaving similarity intact. This extends to recurrent architectures and to deep neural networks, where continued training beyond performance saturation produces activity drift while preserving representational similarity. Although connectivity in the brain is not random, networks trained on high-dimensional inputs acquire connectivity that behaves statistically like a random projection, making these results broadly applicable to biological neural circuits.