Faces and bodies are increasingly integrated along the visual hierarchy in humans and deep neural networks

Abstract

Human visual cortex contains regions specialized for faces and bodies, yet we perceive people as a whole. Why does the brain appear to segregate faces and bodies, and how are they integrated to support person perception? Here, we test whether deep neural network models optimized for visual recognition develop segregated or integrated face and body processing, and how this compares to fMRI activity in visual cortex during natural image viewing. We find that models contain face- and body-selective units but also mixed-selective units that are tuned to both faces and bodies. While face- and body-selective units explain unique variance in their corresponding cortical regions, mixed-selective units best explain activity across regions, and shared variance increases from posterior to anterior cortex. Together, our findings suggest that face and body processing reflects a balance of segregation and integration along the visual hierarchy in humans and models, supporting specialized yet flexible person perception.