Hearing through lip-reading: the brain synthesizes features of absent speech

Abstract

Lip-reading is crucial to understand speech in challenging conditions. Neuroimaging investigations have revealed that lip-reading activates auditory cortices in individuals covertly repeating absent--but known--speech. However, in real-life, one usually has no detailed information about the content of upcoming speech. Here we show that during silent lip-reading of unknown speech, activity in auditory cortices entrains more to absent speech than to seen lip movements at frequencies below 1 Hz. This entrainment to absent speech was characterized by a speech-to-brain delay of 50-100 ms as when actually listening to speech. We also observed entrainment to lip movements at the same low frequency in the right angular gyrus, an area involved in processing biological motion. These findings demonstrate that the brain can synthesize high-level features of absent unknown speech sounds from lip-reading that can facilitate the processing of the auditory input. Such a synthesis process may help explain well-documented bottom-up perceptual effects.