When meaning becomes decodable: Linking neural markers of semantic processing and semantic representations

Abstract

In non-invasive studies of the human brain, semantic processing during language comprehension has been extensively studied using the N400, a component of the electrophysiological evoked response that is strongly modulated by semantic context. More recently, a complementary approach has emerged that uses multivariate pattern analysis to perform neural decoding of semantic vectors from brain activity, operating on the basis that semantic vectors that more closely align with representations in the brain can be more accurately decoded. To consolidate these two approaches, we investigated the relationship between N400 modulation and semantic decoding performance using magnetoencephalography (MEG) in a controlled priming experiment. Twenty-five native speakers of Finnish read word triplets, for which the semantic relatedness between the two primes and the target word was manipulated based on distance in a word2vec embedding space (highly related, moderately related, or unrelated). We found that words presented after unrelated primes elicited higher N400 responses and provided the best examples for training a decoder to map distributed MEG responses to semantic vectors. Semantic information was decodable from approximately 100 to 500 ms after stimulus onset, at all three levels of contextual support. Soon after the N400 peak, neural responses no longer seemed to encode information that could be mapped to context-invariant semantic vectors. This suggests that semantic information is progressively reshaped by linguistic context within and beyond the N400 time window. Semantic decoding therefore offers additional information about representational content that cannot be inferred from the N400 response alone.