Sociality and Interaction Envelope Organize Visual Action Representations

Abstract

Humans observe a wide range of actions in their surroundings. How is the visual cortex organized to process this diverse input? Using functional neuroimaging, we measured brain responses while participants viewed short videos of everyday actions, then probed the structure in these responses using voxel-wise encoding modeling. Responses were well fit by feature spaces that capture the body parts involved in an action and the actions targets (i.e. whether the action was directed at an object, another person, the actor, and space). Clustering analyses revealed five large-scale networks that summarized the voxel tuning: one related to social aspects of an action, and four related to the scale of the interaction envelope, ranging from fine-scale manipulations directed at objects, to large-scale whole-body movements directed at distant locations. We propose that these networks reveal the major representational joints in how actions are processed by visual regions of the brain. Significance StatementHow does the brain perceive other peoples actions? Prior work has established that much of the visual cortex is active when observing others actions. However, this activity reflects a wide range of processes, from identifying a movements direction to recognizing its social content. We investigated how these diverse processes are organized within the visual cortex. We found that five networks respond during action observation: one that is involved in processing actions social content, and four that are involved in processing agent-object interactions and the scale of the effect that these actions have on the world (its "interaction envelope"). Based on these findings, we propose that sociality and interaction envelope size are two of the major features that organize action perception in the visual cortex.