Non-conscious Multisensory Integration in the Ventriloquist Effect

Abstract

The degree to which information from distinct sensory modalities can interact in the absence of conscious awareness remains controversial. According to the Global Neuronal Workspace Theory (GNWT), unconscious sensory information remains relatively confined to sensory cortex and should not be capable of interacting with other modalities until it is broadcast into the (conscious) global workspace comprising late (>300ms) frontal-parietal activation. The ventriloquist effect is a classical multisensory integration phenomenon that refers to the misperception of sound location towards concurrent visual stimulation, such as perceiving the voice of a ventriloquist actor as coming from the moving dummy. Here we used meta-contrast masking to render a brief flash stimulus non-conscious while 30 participants (male, female, and non-binary) performed a sound localization task. We found that, despite being subjectively invisible and at chance at discriminating the flash location, participants were nevertheless biased to localize sounds towards the unconscious flash locations. The effect was present in virtually all participants, was nearly as large as the effect on conscious trials, and was robust to controls for individual differences in task performance. Decoding and ERP analyses of concurrently recorded EEG signals showed that the non-conscious flash location information was present up until around 220ms but not after; suggesting that the visual influence on sound perception may have occurred before conscious broadcast. In support of this, early decoding accuracy on non-conscious trials predicted individual variation in the size of the unconscious ventriloquist effect. Our findings suggest that subjective perception is not required for the integration of signals originating in distinct sensory modalities prompting new questions about the role of subjective perception in multisensory integration.