Exploiting Electrophysiological Measures of Semantic Processing for Auditory Attention Decoding

Abstract

In Auditory Attention Decoding, a users electrophysiological brain responses to certain features of speech are modelled and subsequently used to distinguish attended from unattended speech in multi-speaker contexts. Such approaches are frequently based on acoustic features of speech, such as the auditory envelope. A recent paper shows that the brains response to a semantic description (i.e., semantic dissimilarity) of narrative speech can also be modelled using such an approach. Here we use the (publicly available) data accompanying that study, in order to investigate whether combining this semantic dissimilarity feature with an auditory envelope approach improves decoding performance over using the envelope alone. We analyse data from their Cocktail Party experiment in which 33 subjects attended to one of two simultaneously presented audiobook narrations, for 30 1-minute fragments. We find that the addition of the dissimilarity feature to an envelope-based approach significantly increases accuracy, though the increase is marginal (85.4% to 86.6%). However, we subsequently show that this dissimilarity feature, in which the degree of dissimilarity of the current word with regard to the previous context is tagged to the onsets of each content word, can be replaced with a binary content-word-onset feature, without significantly affecting the results (i.e., modelled responses or accuracy), putting in question the added value of the dissimilarity information for the approach introduced in this recent paper.