Auditory-motor interactions are sensitive to adaptation and learning on multiple timescales

Abstract

Auditory-motor learning is critical for mastering the production of complex sounds, such as speaking and playing music. It is anchored upon internal models of interactions between actions and their sensory consequences, which are fine-tuned by minimizing the errors between the predicted and received sound. Here, we investigated the neural dynamics of sensorimotor learning by manipulating sensorimotor surprisal, with high surprisal defined as unexpected changes in the mapping between actions and their auditory outcomes. Participants performed a piano-playing task in which the key-to-pitch mapping switched unpredictably among three configurations, creating a dynamic sensorimotor environment that required ongoing adaptation. At the change boundaries, a signature of violated motor-to-auditory predictions was found in the auditory evoked responses at N100, which could not be attributed to either purely auditory surprisals or motor execution errors. This prediction error is modulated by short-term context, with greater error responses following longer periods of no map change, indicating that the brain continuously tracks short-term map contexts and rapidly adapts to them. In contrast, 30 minutes of extended goal-directed training on a single key-pitch map modulated P50 amplitude only for the trained map, a modulation that can be explained by a slow, training-driven update of the sensorimotor map. Hence, while auditory predictions from motor actions can be implicitly learned within short-term contexts for rapid adaptation, the complementary process of building a reliable sensorimotor map requires targeted training sustained over time. Our approach of studying auditory-motor surprisal in time-varying sequences reveals that auditory-motor learning is fast, context-sensitive, and shaped by both short- and long-term experience.