An ecological investigation of the effect of background noise on speech processing in a Virtual Classroom

Abstract

Many real-life situations can be extremely noisy. Psychoacoustic studies have shown that background noise can have a detrimental effect on the ability to process and understand speech. However, most studies use stimuli and task designs that are highly artificial, limiting their generalization to more realistic contexts. Moreover, to date, we do not fully understand the neurophysiological consequences of trying to pay attention to speech in a noisy place. To address this lab to real-life gap and increase the ecological validity of speech in noise research, here we introduce a novel audiovisual Virtual Reality (VR) experimental platform. Combined with neurophysiological measurements of neural activity (EEG), eye-gaze and skin conductance (GSR) we studied the effects of background noise in a realistic context where the ability to process and understand continuous speech is especially important: A VR Classroom. Participants (n=32) sat in a VR Classroom and were told to pay attention to mini-lecture segments by a virtual teacher. Trials were either Quiet or contained background construction noise, emitted from outside the classroom window, which was either Continuous (drilling) or Intermittent (air hammers). Result show that background noise had a detrimental effect on learning outcomes, which was also accompanied by reduced neural tracking of the teachers speech. Comparison of the two noise types showed that the intermittent construction noise was more disruptive than continuous noise, as index by both behavioral and neural measures, and it also elicited higher skin-conductance levels, reflecting heightened arousal. Interesting, eye-gaze dynamics were not affected by the presence of noise. This study advances our understanding of the neurophysiological effects of background noise and extends it to more ecologically relevant contexts. It also emphasizes the role that temporal dynamics play for processing speech in noise, highlighting the need to consider the features of realistic noises, as we expand speech in noise research to increasingly realistic circumstances.