When and why modular representations emerge

Abstract

Experimental and theoretical work has argued both for and against the existence of specialized sub-populations of neurons (modules) within single brain regions. By studying artificial neural networks, we show that this local modularity emerges to support complex (for instance, context-dependent) behavior only when the input to the network is low-dimensional. No anatomical constraints are required. We also show when modular specialization emerges implicitly at the population level, where orthogonal subspaces are viewed as distinct modules. Modularity yields abstract representations, allows for rapid learning and generalization on novel output domains as well as related tasks. Non-modular representations facilitate the rapid learning of unrelated tasks. Our findings reconcile conflicting experimental results and make predictions for future experiments.