Metacognition in the audiovisual McGurk illusion: perceptual and causal confidence

Abstract

Almost all decisions in everyday life rely on multiple sensory inputs that can come from common or independent causes. These situations invoke perceptual uncertainty about environmental properties and the signals causal structure. Using the audiovisual McGurk illusion this study investigated how observers formed perceptual and causal confidence judgments in multisensory integration tasks under causal uncertainty. Observers were presented with spoken syllables, their corresponding articulatory lip movements, or their congruent and McGurk combinations (e.g. auditory B/P with visual G/K). Observers reported their perceived auditory syllable, the causal structure and confidence for each judgement. Observers were more accurate and confident on congruent than unisensory trials. Their perceptual and causal confidence were tightly related over trials as predicted by the interactive nature of perceptual and causal inference. Further, observers assigned comparable perceptual and causal confidence to veridical G/K percepts on audiovisual congruent trials and their causal and perceptual metamers on McGurk trials (i.e. illusory G/K percepts). Thus, observers metacognitively evaluate the integrated audiovisual percept with limited access to the conflicting unisensory stimulus components on McGurk trials. Collectively, our results suggest that observers form meaningful perceptual and causal confidence judgments about multisensory scenes that are qualitatively consistent with principles of Bayesian causal inference.