Reverse Spatiotemporal Hierarchy during Cross-modal Memory Recall and Imagery

Recalling past events is often accompanied by mental imagery of those experiences. Based on previous research, this process engages memory- and sensory-related brain areas. However, the underlying spatiotemporal dynamics remain poorly investigated. Here, we used naturalistic videos of audiovisual events and recorded fMRI data during the tasks in which human participants recalled visual contents when hearing associated sounds and recalled sounds when watching silent videos, after they had well memorized the video contents. With time-resolved fMRI multivariate pattern analyses, we observed reverse spatiotemporal hierarchy during the visual memory recall and imagery: the neural activity in primary visual cortex was delayed compared with high-order visual areas. A similar pattern was found during auditory memory recall and imagery, where the bottom-up progression from the mid-level planum temporale to the high-level superior temporal gyrus observed during auditory perception was absent. However, the primary auditory area was not involved, suggesting modality differences in the role of primary sensory areas in corresponding memory recall. We also observed the activity of the hippocampus, the parahippocampal cortex, the retrosplenial cortex, and the precuneus and examined their temporal dynamics. Overall, our study provided both spatial and temporal accounts of neural activity during the cross-modal memory recall and imagery.