Optimality with room to vary: stiff and sloppy modes in a sensory population

Populations of sensory neurons are thought to be shaped by selective pressures for optimal information transmission, yet real neural circuits display substantial variability across stimulus repeats, across time, and between individuals. Reconciling this variability with normative theories requires understanding not only the optimal code, but also how performance changes under perturbations of that code. Here we analyze large-scale recordings from retinal ganglion cells responding to diverse naturalistic movies and spatial noise. We fit low-rank Ising models, recast as interpretable latent variable models, revealing that retinal population activity is well described by a small number of collective modes tightly coupled to the visual stimulus. The spatial receptive fields of the leading latent variables closely align with the principal components of masked natural images, consistent with efficient coding. Perturbation analyses based on the Hessian of the efficient coding objective show that this agreement is concentrated along stiff directions, where small deviations strongly degrade performance, whereas sloppy directions tolerate large variability. Across stimulus ensembles, the same stiff latent modes form a stable backbone of population interactions that generalizes between scenes and supports both efficient stimulus reconstruction and predictive coding of future inputs. These results show how sensitivity to perturbations structures sensory population codes, allowing normative optimality to coexist with rich variability.