Decreasing hearing ability does not lead to improved visual speech extraction as revealed in a neural speech tracking paradigm

Abstract

The use of visual speech is thought to be especially important in situations where acoustics are unclear and in individuals with hearing impairment. To investigate this in a neural speech tracking paradigm, we measured MEG in sixty-seven mid- to old-age individuals during audiovisual (AV), audio-only (A), and visual-only (V) speech in the context of face masks. First, we could extend previous findings by showing that not only in young normal-hearing individuals but also in aging individuals with decreasing hearing ability the brain is superior in neurally tracking the acoustic spectrogram in AV compared to A presentations, especially in multi-speaker situations. The addition of visual lip movements further increases this benefit. Second, we could show that neural speech tracking in individuals with lower levels of hearing ability is affected more by face masks. However, in this population, the effect seems to be a composite of blocked visual speech and distorted acoustics. Third, we could confirm previous findings, that the neural benefit of visual speech varies strongly across individuals. We show that this general individual ability predicts how much people engage in visual speech tracking in difficult AV listening situations. Interestingly, this was not correlated with hearing thresholds and therefore does seem to be a widely used compensatory strategy in the hearing impaired.