Contextual encoding of states and indices for episodic memories make goal-directed action selection flexible, transparent, and efficient

When we want to solve a problem, our brain helps us by recalling related experiences. Furthermore, it can selectively recall only those experiences that fit especially well to the current context and contingencies. In addition, it provides explanations for chosen actions in terms of prior experiences. We show that experimental data on underlying neural codes of the brain, in particular contextual encoding of states and index cells for episodic memories, enable us to port essential aspects of the problem solving strategy of the brain into a simple algorithm. It only requires local online learning in shallow neural networks with low resolution of synaptic weights in order to explore a new environment, and it supports low-latency decision making. Hence it is especially suited for implementation in edge devices. Surprisingly, the performance of this light-weight algorithm on a standard benchmark task, navigation in Markov Decision Processes, is competitive with that of commonly used but computationally more demanding, less flexible, and less transparent reinforcement learning approaches.