A unifying computational account of temporal context effects in language across the human cortex

Abstract

Deep learning advances have revolutionized computational modeling approaches in neuroscience. However, their black-box nature makes it challenging to use deep learning models to discover new insights about brain function. Focusing on human language processing, we propose a new framework to improve the quality and interpretability of the inferences we make from deep learning-based models. First, we add interpretable components to a deep language model and use it to build a predictive encoding model. Then, we use the models predictive abilities to simulate brain responses to controlled stimuli from published experiments. We find that our model, based on a multi-timescale recurrent neural network, captures many previously reported temporal context effects in human cortex. Its failure to capture other effects also highlights important gaps in current language models. Finally, we use this new framework to generate model-based evidence that supports the proposal that different linguistic features are represented at different timescales across cortex.