Maximizing memory capacity in heterogeneous networks

Abstract

A central problem in neuroscience is identifying the features of neural networks that determine their memory capacity and assessing whether these features are optimized in the brain. In this study, we estimate the capacity of a general class of network models. Our derivation extends previous theoretical results, which assumed homogeneous connections and coding levels (i.e., activation rates of the neurons in memory patterns), to models with arbitrary architectures (varying constraints on the arrangement of connections between cells) and heterogeneous coding levels. Using our analytical results, we estimate the memory capacity of two types of brain-inspired networks: a general class of heterogeneous networks and a two-layer model simulating the CA3-Dentate Gyrus circuit in the hippocampus, known to be crucial for memory encoding. In the first case, we demonstrate that to maximize memory capacity, the number of inward connections and the coding levels of neurons must be correlated, presenting a normative prediction that is amenable to experimental testing. In the second case, we show that memory capacity is maximized when the connectivity and coding levels are consistent with the formation of memory "indices" in the Dentate Gyrus, which bind features in the CA3 layer. This suggests specific neural substrates for the hippocampal index theory of memory encoding and retrieval.