bioRxivNeurosciencePreprint
Learning to live with Dale's principle: ANNs with separate excitatory and inhibitory units
Jonathan H. CornfordiD, Damjan Kalajdzievski, Marco LeiteiD, Amélie Lamarquette, Dimitri Michael KullmanniD, Blake A. RichardsiD
Abstract
AO_SCPLOWBSTRACTC_SCPLOWThe units in artificial neural networks (ANNs) can be thought of as abstractions of biological neurons, and ANNs are increasingly used in neuroscience research. However, there are many important differences between ANN units and real neurons. One of the most notable is the absence of Dales principle, which ensures that biological neurons are either exclusively excitatory or inhibitory. Dales principle is typically left out of ANNs because its inclusion impairs learning. This is problematic, because one of the great advantages of ANNs for neuroscience research is their ability to learn complicated, realistic tasks. Here, by taking inspiration from feedforward inhibitory interneurons in the brain we show that we can develop ANNs with separate populations of excitatory and inhibitory units that learn just as well as standard ANNs. We call these networks Dales ANNs (DANNs). We present two insights that enable DANNs to learn well: (1) DANNs are related to normalization schemes, and can be initialized such that the inhibition centres and standardizes the excitatory activity, (2) updates to inhibitory neuron parameters should be scaled using corrections based on the Fisher Information matrix. These results demonstrate how ANNs that respect Dales principle can be built without sacrificing learning performance, which is important for future work using ANNs as models of the brain. The results may also have interesting implications for how inhibitory plasticity in the real brain operates.
Abstract summary
Claude Opus 5.5 · xhigh effort · 18 Oct 2026 · from the abstract · CC BY-NC-ND 4.0
Splitting network units into excitatory and inhibitory populations need not cost learning: initialize inhibition to centre and standardize excitatory activity, and scale inhibitory updates by Fisher information.
Artificial neural networks (ANNs) are used as models of brain computation, yet their units act as excitatory and inhibitory at once, and imposing the biological separation has normally hurt learning. Inspired by feedforward inhibitory interneurons, the authors report networks with distinct excitatory and inhibitory populations that match standard networks, given a particular initialization and a Fisher-information correction for inhibitory parameters.
Key results
SuggestedNetworks obeying the excitatory/inhibitory separation are reported to learn as well as unconstrained networks.
Stated as a demonstration for the proposed Dale's ANNs (DANNs); the abstract names no task, architecture, dataset or accuracy figure for the comparison. · Abstract
ClaimedThe architecture is framed as a normalization scheme, which fixes a useful initialization.
Inhibition can be initialized so that it centres and standardizes the activity of the excitatory units; no ablation separating this choice from the rest of the recipe is reported. · Abstract
ClaimedInhibitory parameters need their own update scaling for training to work.
Updates to inhibitory unit parameters are corrected using the Fisher Information matrix; the abstract gives no learning curves or results without the correction. · Abstract
Methods, data and reproducibility
Artificial networks are constrained so each unit is purely excitatory or purely inhibitory, with the inhibitory units modelled on feedforward inhibitory interneurons in the brain. Two ingredients make them trainable: an initialization in which inhibition centres and standardizes excitatory activity, and Fisher Information matrix corrections applied when scaling updates to inhibitory parameters.
- Methods & tools
- Dale's ANNs (DANNs)
- Reproducibility gaps
- No code, task, dataset or training configuration is identified in the abstract, so the comparison against standard networks cannot be repeated.
Limitations (3)
- The authors present the relevance to inhibitory plasticity in real brains as a possible implication, not as something tested here. · stated by the authors
- No tasks, architectures or performance numbers are reported, so the claim of matching standard networks cannot be checked or sized. · noted in this summary
- The two enabling ingredients are described together, with no ablation separating the contribution of the initialization from that of the Fisher-based update scaling. · noted in this summary
Read this if you build biologically constrained network models and need excitatory and inhibitory units kept separate without giving up trainability.
- Citations
- 43
Reproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- Learning better with Dale's Law: A Spectral Perspective
Pingsheng Li, Jonathan H. Cornford, … Blake Richards·neural information processing systemsPeer-reviewed·Jan 1, 2023
- Going Beyond the Point Neuron: Active Dendrites and Sparse Representations for Continual Learning
- Brain-like learning with exponentiated gradients
Jonathan H. Cornford, Roman Pogodin, … Blake A. Richards·bioRxiv·Oct 25, 2024
- Cellular and subcellular specialization enables biology-constrained deep learning
Alessandro R. Galloni, Ajay Peddada, … Aaron D. Milstein·Cell ReportsPeer-reviewed·Mar 26, 2026
- Dendritic normalisation improves learning in sparsely connected artificial neural networks
Alex D. Bird, Peter Jedlička, Hermann Cuntz·PLoS Computational BiologyPeer-reviewed·Aug 9, 2021
- A rapid and efficient learning rule for biological neural circuits
Eren Sezener, Agnieszka Grabska‐Barwińska, … Peter E. Latham·bioRxiv·Mar 10, 2021