A hierarchical computational motif unifies neural dynamics across the ventral visual stream

Abstract

Neural representations within individual visual cortical areas are dynamic; they evolve over tens to hundreds of milliseconds, even in response to static images. These dynamics have typically been described as area-specific phenomena with unique computational signatures. Here, we show that representational dynamics across the ventral visual stream follow a common motif: representations within each area shift over time along the same complexity axis that organizes hierarchical visual areas. The spatiotemporal signatures of this shift indicate that it is broadly distributed across the neural population, rather than concentrated in specific subpopulations. These shifts are functionally consequential, allowing for the recognition of more complex images over the course of the response. Further, we show evidence in all visual areas of a 30 ms within-area predictivity signal whose properties are consistent with local recurrence, which may be driving the representational shifts. Yet, we show that current state-of-the-art dynamic models, including one with built-in local recurrent processing, fail to recapitulate the measured neural dynamics. Together, these results reveal a common temporal motif across the ventral hierarchy, suggest local recurrence as a possible driver, and provide a concrete dynamic target for future models of the ventral visual stream.