The Intrinsic Manifold of Spontaneous Activity Constrains Cortical Responses to Naturalistic Stimuli

Abstract

The cerebral cortex constrains its spontaneous activity to a low-dimensional manifold learnable from resting state functional magnetic resonance imaging (fMRI) data. However, it remains unclear whether this intrinsic manifold also captures cortical responses to complex, naturalistic stimuli. To test this, we pretrained a deep variational autoencoder model on 3-Tesla resting-state fMRI data to learn the latent structure of spontaneous activity, and then applied this model without finetuning to 7-Tesla fMRI data acquired during movie-watching. Despite the different field strengths, the model generalized robustly from resting to movie-watching states. The latent representation of stimulus-evoked responses was confined to a subspace that occupied about 13% of the latent space spanned by spontaneous activity, demonstrating that task-related neural responses do not require a distinct representational space. By representing cortical dynamics as an evolving latent trajectory, we found striking differences across individuals or between brain states. During movie watching, the velocity of the latent trajectory provided a reliable marker of cortical engagement, and its temporal structure was highly reliable and sensitive to naturalistic events. These findings suggest that the intrinsic manifold of spontaneous activity forms a full reservoir of cortical states that the brain can differentially engage when interacting with the external environment.