Robustness in Hopfield neural networks with biased memory patterns

Abstract

Biological neural networks are able to store and retrieve patterns in the presence of different types of noise. Hopfield neural networks have been inspired by biological neural networks and provide a model for auto-associative memory patterns. An important parameter in these networks is the pattern bias, i.e. the mean activity level of the network, which is closely related to the degree of sparseness in the coding scheme. Here we studied the relation between robustness against different types of biologically-motivated noise and pattern bias. To do so, we developed performance and robustness measures, which are applicable to varying degrees of sparseness of the memory representations, using analytically-optimized thresholds and corruption tolerances adjusted by mutual information. We then applied these tools in numerical simulations and found that, for different types of noise, the pattern load, i.e. the number of patterns that the network has to store, determined which pattern bias is most robust. Across different types of noise, the higher the pattern load was, the more biased was the most robust performing pattern representation. Given the variation in the sparseness level in different brain regions, our findings suggest that memory pattern encoding schemes (i.e. degree of sparseness) in different brain regions might be adapted to the expected memory load in order to best mitigate the adverse effects of disruptions.