Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?
Martin SchrimpfiD, Jonas KubiliusiD, Ha HongiD, Najib J. MajajiD, Rishi RajalinghamiD, Elias B. IssaiD, Kohitij KariD, Pouya BashivaniD, Jonathan Prescott-Roy, Franziska Geiger, Kailyn SchmidtiD, Daniel L. K. YaminsiD, James J. DiCarloiD
Abstract
The internal representations of early deep artificial neural networks (ANNs) were found to be remarkably similar to the internal neural representations measured experimentally in the primate brain. Here we ask, as deep ANNs have continued to evolve, are they becoming more or less brain-like? ANNs that are most functionally similar to the brain will contain mechanisms that are most like those used by the brain. We therefore developed Brain-Score - a composite of multiple neural and behavioral benchmarks that score any ANN on how similar it is to the brains mechanisms for core object recognition - and we deployed it to evaluate a wide range of state-of-the-art deep ANNs. Using this scoring system, we here report that: (1) DenseNet-169, CORnet-S and ResNet-101 are the most brain-like ANNs. (2) There remains considerable variability in neural and behavioral responses that is not predicted by any ANN, suggesting that no ANN model has yet captured all the relevant mechanisms. (3) Extending prior work, we found that gains in ANN ImageNet performance led to gains on Brain-Score. However, correlation weakened at [≥] 70% top-1 ImageNet performance, suggesting that additional guidance from neuroscience is needed to make further advances in capturing brain mechanisms. (4) We uncovered smaller (i.e. less complex) ANNs that are more brain-like than many of the best-performing ImageNet models, which suggests the opportunity to simplify ANNs to better understand the ventral stream. The scoring system used here is far from complete. However, we propose that evaluating and tracking model-benchmark correspondences through a Brain-Score that is regularly updated with new brain data is an exciting opportunity: experimental benchmarks can be used to guide machine network evolution, and machine networks are mechanistic hypotheses of the brains network and thus drive next experiments. To facilitate both of these, we release Brain-Score.org: a platform that hosts the neural and behavioral benchmarks, where ANNs for visual processing can be submitted to receive a Brain-Score and their rank relative to other models, and where new experimental data can be naturally incorporated.
02 /Specification & Classification
Click any tag to browse matching papers- Citations
- 461
- Organism
- Research Focus
- Approach
In-depth summary
Claude Opus 5.5 · xhigh effort · 17 Oct 2026 · from the full text of version 2 (2 Jan 2020) · CC BY 4.0
Among dozens of ImageNet-trained networks scored against macaque V4/IT recordings and human behavior, DenseNet-169, CORnet-S and ResNet-101 rank highest, and brain similarity stops tracking accuracy above 70% top-1.
Early deep networks resembled primate ventral-stream representations, but it was unclear whether newer, more accurate ImageNet models keep becoming more brain-like. Brain-Score replaces one-off lab comparisons with a standardised, extensible composite of neural and behavioral benchmarks, plus an open platform where models and new data can be submitted.
Key results
ShownOn a composite of macaque V4 and IT neural predictivity and human behavioral consistency, DenseNet-169, CORnet-S and ResNet-101 are the most brain-like models tested, while the most accurate ImageNet model ranks far lower.
Brain-Score .549 for DenseNet-169, .544 for CORnet-S and .542 for ResNet-101; PNASNet, the best ImageNet model tested (82.90% top-1), ranks only 13th. VGG-19 leads on V4 (.672) and DenseNet-169 on IT (.604). · Ranking of state-of-the-art networks; Scores on individual neural and behavioral…
ShownBrain-Score rises with ImageNet accuracy only up to roughly 70% top-1 and shows no significant relationship beyond it, although across the full range accuracy still correlates with each component benchmark.
Below 70% top-1, correlation with Brain-Score .92 (p < 10−14); above 70%, no significant correlation (p >> .05). Across all models, accuracy correlates .68 with V4 (p < 10−28), .80 with IT (p < 10−47) and .93 with behavior (p < 10−91). · Ranking of state-of-the-art networks; Scores on individual neural and behavioral…
ShownEvery model falls short of the data's internal-consistency ceiling on all three benchmarks, so much reliable neural and behavioral variance remains unexplained by any network.
Best V4 predictivity .663 vs ceiling .892; best IT predictivity .604 vs ceiling .817; best behavioral predictivity .378 vs ceiling .497. · Scores on individual neural and behavioral…
ShownSmall or shallow networks can be nearly as brain-like as large high-accuracy ones, especially for IT, which points to simpler models of the ventral stream.
Lightweight BaseNets (six convolutional layers, 47.64% top-1) predict IT at .592 vs .606 for DenseNet-169 (75.90% top-1), with MobileNets at .590 (69.80% top-1); the best BaseNet Brain-Score is .500. CORnet-S, a shallow recurrent network of four areas, ranks second (.544). Inception versions lost Brain-Score as their ImageNet accuracy improved. · Abstract; Candidate Models; Ranking of state-of-the-art networks; Scores on individual neural and behavioral…
ShownBehavioral consistency tracks accuracy overall, yet the highest-accuracy models are less human-like than ResNet-101, and image-level mismatches with humans run in both directions.
ResNet-101 scores .389 on behavior at 77.37% top-1 vs .351 for PNASNet at 82.90%. Of 5,520 images, PNASNet and humans agree within 1 d′ on 3,388 (61.38%); 1,918 (34.75%) are easier for humans and 214 (3.88%) easier for the model, which lowers its behavioral score. · Scores on individual neural and behavioral…
Methods, data and reproducibility
Layer activations of each ImageNet-trained network (PCA-reduced to 1,000 dimensions, best layer per region) are mapped to macaque V4 and IT responses by 25-component partial least squares regression and scored by cross-validated Pearson correlation; pre-readout features predict human image-by-image choice patterns (I2n). Brain-Score is the mean of the V4, IT and behavioral scores.
- Macaque array recordings: responses to grayscale naturalistic object images (eight categories of eight objects), about 50 repeats per image, firing rate 70-170 ms after onset · macaque (two monkeys) · n = 88 V4 and 168 IT neurons; 2,560 images
- Human two-alternative object-choice responses to images shown for 100 ms on Amazon Mechanical Turk, 24 object categories · human · n = 1,472 observers; 240 of 2,400 images x 24 objects
- Candidate ImageNet models: AlexNet, VGG, ResNet, Inception, InceptionResNet, SqueezeNet, DenseNet, MobileNet, (P)NASNet, in-house BaseNets and CORnet-S · n = dozens of models
- Methods & tools
- Brain-Score, Brain-Score.org platform, CORnet-S, BaseNets, partial least squares regression, principal components analysis, I2n behavioral consistency metric, TensorFlow (Keras, Slim), PyTorch, Amazon Mechanical Turk, ImageNet
- Data & accessions
- https://brain-score.org/
- Software & environment
- Pre-trained models came from TensorFlow (Keras or Slim) and, for AlexNet, SqueezeNet, ResNet-18 and ResNet-34, from PyTorch; no library versions are given.
- Reproducibility gaps
- Both links are printed without a scheme. No software versions or hardware are stated, the neural and behavioral datasets come from earlier studies whose citations are lost in this text, and rankings are said to be updated regularly, so these scores describe this version.
Limitations (6)
- The benchmark set is admitted to be incomplete: only V4, IT and one behavioral task on synthetic object-on-background images, with no early visual areas, fMRI, LFP or EEG/MEG data, anatomy or temporal dynamics. · stated by the authors
- Neural ceilings reflect individual neurons from only two monkeys and are just an upper bound, so it is unclear whether models are truly plateauing or the datasets are limiting. · stated by the authors
- Brain-Score shows that one model beats another but not why, implicitly expects one model to fit both macaques and humans, and cannot yet be used to train models. · stated by the authors
- The composite is an unnormalised mean of three scores, which may penalise components with low variance. · stated by the authors
- The top three Brain-Scores (.549, .544, .542) are separated by small margins and given without confidence intervals or between-model tests, so the ranking may not be robust. · noted in this summary
- Some best scores differ between passages: ResNet-101 behavior .378 vs .389, DenseNet-169 IT .604 vs .606, and best V4 .672 (VGG-19) vs .663. · noted in this summary
Who should read:Read this if you compare deep networks with neural or behavioral recordings and want a standardised, extensible scoring scheme plus evidence on whether ImageNet accuracy predicts brain similarity.
Reproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- CORnet: Modeling the Neural Mechanisms of Core Object Recognition
Jonas Kubilius, Martin Schrimpf, … James J. DiCarlo·bioRxiv·Sep 4, 2018
- Brain hierarchy score: Which deep neural networks are hierarchically brain-like?
Soma Nonaka, Kei Majima, … Yukiyasu Kamitani·iSciencePeer-reviewed·Aug 21, 2021
- Comparing Brain-Score and ImageNet performance with responses to the scintillating grid illusion
M. Kraus, Lucy Verkerk, Sander Wessel Keemink·bioRxiv·Jun 18, 2025
- Large-scale, high-resolution comparison of the core visual object recognition behavior of humans, monkeys, and state-of-the-art deep artificial neural networks
Rishi Rajalingham, Elias B. Issa, … James J. DiCarlo·Journal of NeurosciencePeer-reviewed·Jul 13, 2018
- Deep neural networks: a new framework for modelling biological vision and brain information processing
Nikolaus Kriegeskorte·bioRxiv·Oct 26, 2015
- To find better neural network models of human vision, find better neural network models of primate vision
Kamila Maria Jozwik, Martin Schrimpf, … James J. DiCarlo·bioRxiv·Jul 2, 2019