Working Memory as Programmable Fast Weight Computation

Abstract

Working memory (WM) stores information after sensory input disappears and later retrieves it in a task-relevant format, but the mechanism unifying storage and retrieval remains unclear. Here we combine neural geometry analyses of macaque dorsolateral prefrontal cortex activity during a visuospatial delayed-match-to-sample task with computational modeling to test whether WM can be implemented as recurrent fast-weight computation. We found that the relational geometry of remembered locations was strongly expressed during sample presentation, degraded during the early delay, and reemerged before requirement in a partially distinct mnemonic subspace. A recurrent fast-weight programmer model, which implements a form of dynamic fast-weight memory closely related to linear Transformer computation, reproduced these latent-to-mnemonic dynamics. Direct inspection and perturbation of the model revealed that neural activity writes stimulus information into rapidly modifiable synaptic states, synaptic dynamics organize this latent memory over time, and recurrent readout queries the evolving state to generate task-relevant activity. These findings provide a unified account of WM storage and retrieval and suggest that biological WM and Transformer family architectures share an algorithmic principle of programmable temporary memory.