A Deep Learning Framework for Spatiotemporal Modeling of Visual Task fMRI

Abstract

Characterizing the dynamic coordination of distributed brain regions during cognitive tasks remains challenging, as traditional fMRI analysis focuses on localized activations without revealing the underlying information flow that drives them. Here, we propose STREAM (Spatiotemporal Representation for Effective connectivity Analysis Model), a deep-learning framework that learns neural transition functions in task-fMRI to characterize effective connectivity and whole-brain information flow. Applied to visual category processing in 1074 participants, STREAM accurately reconstructs activation maps while further revealing that traditional activation regions are primarily driven by incoming signals. Moreover, the Default Mode Network acts as a high-level regulatory hub with extensive outgoing influence, challenging its passive characterization. Additionally, category-specific communication emerges from dynamic reconfiguration of signaling patterns among key hubs rather than static pathways. These findings establish a novel computational paradigm that uncovers directional signaling mechanisms driving local dynamics in task-fMRI, revealing how the brain flexibly reconfigures functional architecture for complex cognition.