Text-related functionality of visual human pre-frontal activations revealed through neural network convergence

Abstract

The functional role of visual activations of human pre-frontal cortex remains a deeply debated question. Its significance extends to fundamental issues of functional localization and global theories of consciousness. Here we addressed this question by comparing, dynamically, the potential parallels between the relational structure of prefrontal visual activations and visual and textual-trained deep neural networks (DNNs). The frontal visual relational structures were revealed in intra-cranial recordings of human patients, conducted for clinical purposes, while the patients viewed familiar images of faces and places. Our results reveal that visual relational structures in frontal cortex were, surprisingly, predicted by text and not visual DNNs. Importantly, the temporal dynamics of these correlations showed striking differences, with a rapid decline over time for the visual component, but persistent dynamics including a significant image offset response for the text component. The results point to a dynamic text-related function of visual prefrontal responses in the human brain.