Lateral recurrence dynamically refines visual representations for task performance

Abstract

Recurrent connections are pervasive in visual cortex, and visual representations evolve as recurrent processing unfolds. Yet the mechanism underlying this temporal transformation remains poorly understood. To address this question, we trained recurrent convolutional networks on face and object recognition and tested them on a parametric stimulus set in which category, identity, and pose were systematically varied. This design isolates how lateral recurrence reshapes neural-population representational geometry and single-unit tuning across recurrent timesteps. Within the trained domain, recurrence increased the separability between identities in deeper layers, while leaving the non-trained domain representations largely unchanged. Gating lateral weights abolished the across-timestep gain in trained-domain identity decoding, demonstrating that lateral recurrence drives this refinement. At the single-unit level, recurrence strengthened the correlation between each unit's category preference and its within-domain identity sensitivity, providing a unit-level mechanism for the population-level changes. These results identify lateral recurrence as a dynamic mechanism for refining learned representational structure within a fixed neural population, yielding testable predictions for time-resolved recordings in domain-selective cortex.