PolyRNN: A time-resolved model of polyphonic musical expectations aligned with human brain responses

Abstract

Musical expectations shape how we perceive and process music, yet current computational models are limited to monophonic or simplified stimuli. The study of the neural processes underlying musical expectations in real-world music therefore requires significant advances in our statistical modeling of these stimuli. We present PolyRNN, a recurrent neural network designed to model expectations in naturalistic, polyphonic music. We recorded neurophysiological activity non invasively (MEG) and within the human brain (intracranial EEG) while participants listened to naturally expressive piano recordings. The musical expectations estimated by the model are encoded in evoked P2- and P3-like components in auditory regions. Comparing PolyRNN to a state-of-the-art generative music model, we show that piano roll representations are best suited to represent expectations in polyphonic contexts. Overall, our approach provides a new way to capture the musical expectations emerging from natural music listening, and enables the study of predictive processes in more ecologically valid settings.