Stochastic activity in low-rank recurrent neural networks

Abstract

The geometrical and statistical properties of brain activity depend on the way neurons connect to form recurrent circuits. However, the link between connectivity structure and emergent activity remains incompletely understood. We investigate this relationship in recurrent neural networks with additive stochastic inputs. We assume that the synaptic connectivity can be expressed in a low-rank form, parameterized by a handful of connectivity vectors, and examine how the geometry of emergent activity relates to these vectors. Our findings reveal that this relationship critically depends on the dimensionality of the external stochastic inputs. When inputs are low-dimensional, activity remains low-dimensional, and recurrent dynamics influence it within a subspace spanned by a subset of the connectivity vectors, with dimensionality equal to the rank of the connectivity matrix. In contrast, when inputs are high-dimensional, activity also becomes potentially high-dimensional. The contribution of recurrent dynamics is apparent within a subspace spanned by the totality of the connectivity vectors, with dimensionality equal to twice the rank of the connectivity matrix. Applying our formalism to excitatory-inhibitory networks, we discuss how the input configuration also plays a crucial role in determining the amount of amplification generated by non-normal dynamics. Our work provides a foundation for studying activity in structured brain circuits under realistic noise conditions, and offers a framework for interpreting stochastic models inferred from experimental data.