High-dimensional population codes reveal interpretable and diverse features underlying visual perception

Abstract

Despite large-scale recordings, neuroscience has identified representations for only a small fraction of visual features distinguishable by humans. This is increasingly attributed to mixed selectivity, where individual neurons respond to multiple stimuli, requiring population-level readouts of human-recognizable ('interpretable') representations. To identify interpretable representations at scale, we 1)computationally extract population codes from macaque electrophysiology datasets spanning V4 and IT cortex and vision models and 2) introduce an automated method to measure representation interpretability and diversity, then isolate a set of unique, meaningful features encoded by neurons versus populations. Across datasets, populations represent more interpretable, diverse features than neurons. This advantage grows with the number of sampled neurons and image diversity. Finally, we demonstrate through targeted ablations that interpretable representations play a causal role in downstream behavior. Overall, our framework enables a more comprehensive account of the features represented across biological and artificial visual systems and their contributions to visual perception.