Bottom-up and generative computations uniquely explain neural responses across the social brain

Abstract

Making social evaluations from visual input is a core human ability that engages brain regions involved in social perception, including portions of the superior temporal sulcus (STS), as well as higher-level mentalizing regions, such as the temporoparietal junction (TPJ). One common hypothesis proposes that these regions operate hierarchically: social perception regions like posterior STS (pSTS) implement bottom-up computations to generate fast, stimulus-derived representations of social interactions, while mentalizing regions like TPJ perform inverse-planning computations to infer the underlying goals and motivations driving agents behavior. However, this computational-neural mapping has never been formally tested, in large part due to the lack of successful computational models of social processing. We developed computational models aligned with these two frameworks: a graph-neural-network (GNN) model that recognizes social interactions by relying on relational visual information, and a generative inverse-planning model that does so by inverting a model of agents goals and the physical world. In this preregistered study, we collected fMRI responses while participants watched videos of agentive animated shapes depicting social interactions and compared neural responses to both computational models. Surprisingly, we found that both the GNN and inverse-planning model explained neural responses in pSTS and TPJ, even after controlling for variance explained by the other model. Exploratory analyses, however, revealed a shift from early perceptual processing towards later higher-order reasoning in both regions, suggesting a temporal rather than spatial hierarchy. Overall, this study provides the first evidence that both social perception and mentalizing regions carry out a combination of relational bottom-up and higher-level inferential computations, perhaps on distinct timescales. This work also provides the first comparison of an inverse-planning model to neural activity and demonstrates that theory-driven cognitive models can successfully predict fMRI responses to social scenes. Significance StatementThe ability to recognize social interactions between others is central to humans daily lives and engages brain regions supporting social perception and mental state inference. The neural computations underlying this ability, however, are poorly understood. Here we leveraged new models of bottom-up social perception and generative social inference to test the hypothesis that these complementary computations are carried out in separate brain regions. We compared both models to brain responses from subjects viewing procedurally generated videos of social interactions. Surprisingly, we found that both models explained neural activity in both perceptual and mentalizing regions, even when controlling for effects of the other model. These findings challenge the idea of a strict division of labor in the social brain and refine our understanding of the computations supporting human social inference.