The transformer as a candidate canonical cortical microcircuit

Abstract

Neuroscience hypothesizes a canonical microcircuit subserving cortical computation across brain systems. In artificial intelligence, neural networks have become a universal computational substrate. Here, we leverage newly available large-scale data and mapping techniques to compare neural network motifs to the brain. We embedded a spectrum of candidate microcircuit architectures within a common spatial macrocircuit, measuring alignment with an extensive battery of cortical signatures. We find that one specific circuit type--the transformer--uniquely captures fine-grained unit-level circuit details, population-level functional maps and neuronal activity patterns, and systems-level wiring efficiency and behavioral demands. The key ingredient enabling the transformer's success is a learned multiplicative component making relational information available without extensive long-range wiring. These results suggest striking convergence between computational motifs in natural and artificial intelligence.