Distinct perceptual and conceptual representations of natural actions along the lateral and dorsal visual streams: an EEG-fMRI fusion study

Abstract

Actions are the building blocks of our dynamic visual world, yet the neural computations supporting action perception are not well understood. How does perceptual and conceptual information unfold in the brain when we observe what others are doing? We collected EEG and fMRI data while participants viewed short videos and sentences depicting naturalistic actions. Using representational similarity analysis, we found a posterior-to-anterior gradient along the lateral pathway, from perceptual features to conceptual, modality-invariant representations. Among conceptual features, the target of actions (i.e. whether the action was directed at an object, a person, or the self) explained the most unique variance in EEG responses. In fMRI, we found distinct conceptual representations along the ventral, dorsal, and lateral pathways, with the target of actions specifically encoded in lateral occipitotemporal cortex (LOTC) and posterior superior temporal sulcus (pSTS). Finally, EEG-fMRI fusion revealed rapid processing along the lateral and dorsal pathways. Together, our results disentangle the perceptual and conceptual components of action understanding and characterize the underlying spatiotemporal dynamics in the human brain.