Sequential Sampling as a Computational Principle of Episodic Memory

Abstract

Visual perception unfolds through sequential eye movements, yet computational models of episodic memory typically assume that event representations already exist. Here, we propose that sequential gaze sampling does not simply supply memory with visual information, but helps construct the representations that are subsequently stored and retrieved. We introduce a computational model in which successive fixation-dependent samples are integrated into an evolving internal representation before one-shot storage in associative memory. Within individual elements, successive samples progressively individuate the representation; across elements, transitions promote relational integration. Crucially, both effects arise from the same history-dependent updating process where each new sample is integrated into a state already shaped by what came before. As a result, sampling history becomes embedded in the memory representation itself. Changing the order of otherwise identical samples alters both the resulting representation and element-specific retrieval, while reinstating the original sampling trajectory facilitates access to the stored memory. Model-neural analysis further show that changes in the model's evolving representations correspond to neural representational dynamics in lateral temporal cortex. Together, these findings support sequential sampling as a candidate computational process for constructing representations available to episodic memory, linking ongoing perceptual sampling to subsequent memory formation and retrieval.