Learning a shared vocabulary between episodic and semantic memory enhances recall and compositional consolidation

Abstract

Semantic knowledge is thought to emerge through the consolidation of episodic experience, yet the biological mechanisms by which reusable semantic representations are extracted from complex episodes remain unclear. Conversely, semantic representations can themselves be found within the medial temporal lobe, raising the questions of how they arise there and why structured semantic overlap should benefit an episodic memory system. We propose that replay establishes a shared semantic vocabulary between MTL and CTX through two complementary processes. First, episodic replay enables neocortex to extract reusable semantic concepts from overlapping episodes via a form of dictionary learning. Second, spontaneous cortical activity replays these learned concepts back to MTL, allowing semantic representations to become part of the episodic code. We implement this framework in a circuit model using local Hebbian plasticity and homeostatic competition. The model reproduces key experimental observations, including concept-cell-like representations, increased overlap between related memories following consolidation, and synaptic reorganization associated with systems consolidation. Functionally, embedding semantic representations within episodic memory improves recall by providing an error-correcting scaffold and enhancing replay consistency, inducing a generalization bias in the consolidation of compositional representations. This further explains experimental effects hard to reconcile with standard accounts of consolidation, such as the impact of disrupting semantics in episodic recall and semantic deficits following medial temporal lobe lesions. More broadly, our results suggest that consolidation progressively builds a shared vocabulary between episodic and semantic memory, allowing structure extracted from past experience to organize the representation and consolidation of future memories.