Sparse coding at the input stage of the hippocampus enables fast learning

Abstract

The hippocampus is crucial for fast learning, a capacity commonly attributed to the sparse code of the dentate gyrus (DG), the input stage of the trisynaptic circuit. However, this has been assumed rather than demonstrated, and we lack a principled account of when sparsity helps or hurts learning and what is traded off along the sparse-to-dense continuum. We characterize sparse coding in neural networks trained on nonlinear tasks and show that the level and position of sparsity determine learning speed and the memorization-generalization trade-off. Learning speed follows an inverted-U function, peaking at capacity-sufficient codes. Critically, sparsity must be implemented at the input stage, mirroring the DG. Sparsity speeds learning by producing high-dimensional, pattern-separated representations, but at a cost: models learn through memorization and fail on new examples. Our work establishes principles linking sparsity to learning, casting the DG's sparse code as a mechanism for episodic snapshots rather than general structure.