Replay as a basis for backpropagation through time in the brain

How episodic memories are formed in the brain is an outstanding puzzle for the neuroscience community. The brain areas that are critical for episodic learning (e.g., the hippocampus) are characterized by recurrent connectivity and generate frequent offline replay events. The function of the replay events is a subject of active debate. Recurrent connectivity, computational simulations show, enables sequence learning when combined with a suitable learning algorithm such as Backpropagation through time (BPTT). BPTT, however, is not biologically plausible. We describe here, for the first time, a biologically plausible variant of BPTT in a reversible recurrent neural network, R2N2, that critically leverages offline-replay to support episodic learning. The model uses forwards and backwards offline replay to transfer information between two recurrent neural networks, a cache and a consolidator,that perform rapid one-shot learning and statistical learning, respectively. Un-like replay in standard BPTT, this architecture requires no artificial external memory store. This architecture and approach outperform existing solutions and account for the functional significance to hippocampal replay events. We demonstrate the R2N2 network properties using benchmark tests from computer science and simulate the rodent delayed alternation T-maze task.