Audio-visual interactions in egocentric distance perception: Ventriloquism effect and aftereffect1

Abstract

The processes of audio-visual integration and of visually-guided re-calibration of auditory distance perception are not well understood. Here, the ventriloquism effect (VE) and aftereffect (VAE) were used to study these processes in a real reverberant environment. Auditory and audio-visual (AV) stimuli were presented, in interleaved trials, over a range of distances from 0.7 to 2.04 m in front of the listener, whose task was to judge the distance of auditory stimuli or of the auditory components of AV stimuli. The relative location of the visual and auditory components of AV stimuli was fixed within a session such that the visual component was presented from distance 30% closer (V-closer) than the auditory component, 30% farther (V-farther), or aligned (V-aligned). The study examined the strength of VE and VAE as a function of the reference distance and of the direction of the visual component displacement, and the temporal profile of the build-up/break-down of these effects. All observed effects were approximately independent of target distance when expressed in logarithmic units. The VE strength, measured in the AV trials, was roughly constant for both directions of visual-component displacement such that, on average, the responses shifted in the direction of the visual component by 72% of the audio-visual disparity. The VAE strength, measured on the interleaved auditory-only trials, was stronger in the V-farther than the V-closer condition (44% vs. 31% of the audio-visual disparity, respectively). The VAE persisted to post-adaptation auditory-only blocks of trials, however it was weaker and the V-farther/V-closer asymmetry was reduced. The rates of build-up/break-down of the VAE were also asymmetrical, with slower adaptation in the V-closer condition. These results suggest that, on a logarithmic scale, the AV distance integration is symmetrical, independent of the direction of induced shift, while the visually-induced auditory distance re-callibration is asymmetrical, stronger and faster when evoked by more distant visual stimuli.