Mapping multi-modal dynamic network activity during naturalistic music listening

The human brain is a complex, adaptive system capable of parsing complex stimuli and generating complex behaviour. Understanding how to model and interpret the dynamic relationship between brain, behaviour, and the environment will provide vital information on how the brain responds to real-world stimuli, develops and ages, and adapts to pathology. Modelling together numerous streams of dynamic data, however, presents sizable methodological challenges. In this paper, we present a novel workflow and sample interpretation of a data set incorporating brain, behavioural, and stimulus data from a music listening study. We use hidden Markov modelling (HMM) to extract state timeseries from continuous high-dimensional EEG and stimulus data, estimate timeseries variables consistent with HMM from continuous low-dimensional behavioural data, and model the multi-modal data together using partial least squares (PLS). We offer a sample interpretation of the results, including a discussion on the limitations of the currently available tools, and discuss future directions for dynamic multi-modal analysis focusing on naturalistic behaviours.