Pupil size and EEG speech tracking as independent measures of listening effort and speech intelligibility

Abstract

Speech is hard to understand when there is background noise. Speech intelligibility and listening effort both affect our ability to understand speech, but the relative contribution of these subjective factors is hard to disentangle. Previous studies suggest that speech intelligibility could be assessed with EEG speech tracking and listening effort via pupil size. However, these measures may be confounded, because poor intelligibility may require a larger effort. To address this we developed a novel word-detection paradigm that allows for a rapid behavioral assessment of speech processing. In this paradigm words appear on the screen during continuous speech similar to closed captioning. In two listening experiments with a total of 51 participants we manipulated intelligibility with changing auditory noise levels and modulated effort by varying monetary reward. Increasing signal-to-noise ratio (SNR) improved detection performance along with EEG speech tracking, suggesting improved intelligibility. Additionally, we find larger pupil size with increased SNR, suggestive of increased effort. Surprisingly, when we modulated both reward and SNR, we found that reward modulated only pupil size, while SNR modulated only EEG speech tracking. We suggest that this new paradigm can be used to independently and objectively assess speech intelligibility and listening effort.