Covert detection of own-name and semantic violations in task-irrelevant speech, in a realistic Virtual Cafe

Abstract

Detecting that someone has said your name is one of the most famous examples for incidental processing of supposedly-unattended speech. However, empirical investigation of this so-called "cocktail party effect" has yielded conflicting results. We present a novel empirical approach for revisiting this effect under highly ecological conditions, by immersing participants in a multisensory virtual cafe environment and using realistic stimuli and tasks. Participants listened to conversational speech from a character sitting across from them, while a barista in the back of the cafe called out food orders. Unbeknownst to them, the barista sometimes called orders containing their own name or semantic violations. We used combined measurements of brain activity (EEG), eye-gaze and galvanic skin response to assess the response-profile to these two probes in the task-irrelevant barista-stream. Both probes elicited unique neural and physiological responses relative to control stimuli, indicating that the system indeed processed these words and detected their unique status, despite being task-irrelevant. Interestingly, these responses were covert in nature and were not accompanied by gaze-shifts towards the barista character. This pattern demonstrates that under these highly ecological conditions, listeners incidentally pick up information from task-irrelevant speech, emphasizing the dynamic and non-binary nature of attention in real-life environments.