When touch leads vision: flexible sensory leadership during active texture exploration

Abstract

Vision can sample objects remotely and over a broad field, whereas touch is restricted to contacted skin and is often spatially serial however might outperform vision in resolution by transforming spatial information to a temporal code. This raises the question on how multisensory exploration is coordinated. We recorded gaze and or touch location in exploration of 60 three-dimensional surfaces with locally varying textures during visual, haptic and bimodal exploration while participants performed a same/different task, judging whether a second texture matched the first. In bimodal exploration, gaze and touch became more spatially similar than during unimodal exploration, while touch changed less than vision between unimodal and bimodal contexts. A temporal overlap analysis further showed that touch preceded gaze on average during the first 5 s of bimodal exploration. We then increased visual discriminability of the textures using cast-shadow illumination. A psychophysical pilot confirmed improved texture discrimination, and a new participant group showed a strong shift towards vision-leading exploration. Thus, sensory leadership during active visuo-haptic exploration is not fixed: touch can lead vision when texture information is important, but the balance adapts when visual information becomes more diagnostic.