Frontal cortex organization supporting audiovisual processing during naturalistic viewing

Abstract

AbstractOur brains dynamically adapt to a multisensory world by orchestrating diverse inputs across sensory streams. This process engages multiple brain regions, but it remains unclear how audiovisual stimuli are represented and evolve over time, especially in naturalistic scenarios. Here, we employed a movie-viewing paradigm to explore this question. We recorded intracranial electrocorticography (iEEG) to measure brain activity in 19 participants watching a short multilingual movie. Using unsupervised clustering and supervised encoding models, we identified a robust modality-specific gradient in the frontal cortex, wherein the ventral division primarily processes auditory information and the dorsal division processes visual inputs. Further, we found that this cortical organization dynamically changed, adapting to different movie contexts. This result potentially reflects flexible audiovisual-resource assignment to construct a coherent percept of the movie. Leveraging behavioral ratings, we found that the frontal cortex is the primary site in this modality assignment process. Together, our findings shed new light on the functional architecture of the frontal cortex underlying flexible multisensory representation and integration in natural contexts.