High-level Prediction of Continuous Speech During Mind-Wandering

Abstract

Abundant evidence shows that when listening to speech or reading text, we continuously make predictions about upcoming words. Does this process stop when our mind wanders away and subjective experience becomes decoupled from the content of the speech? Conversely, is word prediction sufficient for conscious experience, or can these processes dissociate? Participants (N = 25) listened to audiobook content (> 12,000 words) while providing self-reports of mind-wandering. Mind-wandering was associated with spectral changes in the EEG signal, compared to attentive listening, as well as decreased early response to word onset, in line with current accounts. In contrast, EEG markers of word-level contextual surprise (indexed by the contextual surprise of large language models) remained intact during mind-wandering, alongside significant encoding of semantic content. Last, we measured the neural encoding of predictive context, measured by the correlation between EEG activity and the vectorial embeddings that language models use to predict each word given prior context. This encoding also persisted during mind-wandering, albeit weaker compared to attentive periods. Our findings show that the brain predicts and monitors relevant inputs even when the subjective experience of the external environment is dimmed during mind wandering. This suggests that the shared computational mechanisms between humans and Language Models are insufficient for having a subjective experience of the meaning of speech contents, with implications for theoretical accounts of mind-wandering and predictive processing accounts of consciousness. Significance StatementLarge Language Models (LLMs) achieve human-like conversational abilities by using long range context to predict the most likely next word. Such predictive processing is also common in the brain, and some link the best prediction to the current conscious experience. Using EEG, we show that next-word prediction markers, marked by LLM-brain alignment, persist during periods of mind-wandering while people listen to audiobooks. Specifically, we found that when people report mind-wandering, their brains continue to represent context-based predictions, word-level surprisal, and word-level semantics. These findings dissociate high-level predictions from subjective experience.