A deep residual model for characterization of 5D spatiotemporal network dynamics reveals widespread spatiodynamic changes in schizophrenia

Schizophrenia is a severe brain disorder with serious symptoms including delusions, disorganized speech, and hallucinations that can have a long-term detrimental impact on different aspects of a patients life. It is still unclear what the main cause of schizophrenia is, but a combination of altered brain connectivity and structure may play a role. Neuroimaging data has been useful in characterizing schizophrenia, but there has been very little work focused on voxel-wise changes in multiple brain networks over time, despite evidence that functional networks exhibit complex spatiotemporal changes over time within individual subjects. Recent studies have primarily focused on static (average) features of functional data or on temporal variations between fixed networks; however, such approaches are not able to capture multiple overlapping networks which change at the voxel level. In this work, we employ a deep residual convolutional neural network (CNN) model to extract 53 different spatiotemporal networks each of which captures dynamism within various domains including subcortical, cerebellar, visual, sensori-motor, auditory, cognitive control, and default mode. We apply this approach to study spatiotemporal brain dynamism at the voxel level within multiple functional networks extracted from a large functional magnetic resonance imaging (fMRI) dataset of individuals with schizophrenia (N=708) and controls (N=510). Our analysis reveals widespread group level differences across multiple networks and spatiotemporal features including voxel-wise variability, magnitude, and temporal functional network connectivity in widespread regions expected to be impacted by the disorder. We compare with typical average spatial amplitude and show highly structured and neuroanatomically relevant results are missed if one does not consider the voxel-wise spatial dynamics. Importantly, our approach can summarize static, temporal dynamic, spatial dynamic, and spatiotemporal dynamics features, thus proving a powerful approach to unify and compare these various perspectives. In sum, we show the proposed approach highlights the importance of accounting for both temporal and spatial dynamism in whole brain neuroimaging data generally, shows a high-level of sensitivity to schizophrenia highlighting global but spatially unique dynamics showing group differences, and may be especially important in studies focused on the development of brain-based biomarkers.