henhaobio
Sign in

bioRxivNeurosciencePreprint

CORnet: Modeling the Neural Mechanisms of Core Object Recognition

Jonas KubiliusiD, Martin SchrimpfiD, Aran NayebiiD, Daniel M. Bear, Daniel L. K. YaminsiD, James J. DiCarloiD

PDFDOICode

In-depth summary

Claude Opus 5.5 · xhigh effort · 8 Nov 2026 · from the full text of version 1 (4 Sept 2018) · CC BY 4.0

CORnet-S, a four-area recurrent network with areas pre-mapped to V1, V2, V4 and inferior temporal cortex, ranks among top Brain-Score models while having a shorter longest convolutional path than competitors.

Deep convolutional networks are the best current models of the primate ventral visual stream, but the strongest ones now have more than 100 processing levels against 4-8 cortical levels, which makes it hard to say which layers correspond to V1, V2, V4 or inferior temporal cortex (IT), and they are purely feedforward. CORnet fixes four areas mapped in advance to those regions and adds within-area recurrence and skip connections, tracking both ImageNet performance and the fit to primate neural and behavioral data.

Key results

  1. ShownCORnet-S, which adds a ResNet-style bottleneck and skip connection to a within-area recurrent circuit, scores among the top models on Brain-Score and is simpler than the other top scorers by the authors' Feedforward Simplicity measure.

    Fig. 3 places CORnet-S on a par with the top-scoring networks, a group led by very deep feedforward models such as ResNet-101 and DenseNet-169, and gives it the highest simplicity among them. Each CORnet-S area has a path length of four convolutions, with recurrent paths counted once. The text gives no numeric Brain-Score values. · Brief history of modeling vision; Feedforward Simplicity; CORnet-S is one of the best yet substantially…; Figure legends

  2. ShownCORnet-Z, a purely feedforward model with one convolution, ReLU and max pooling per area, is a lighter alternative to AlexNet with similar brain and behavior prediction.

    It reaches 48% ImageNet top-1 after under 20 hours of training on one Titan X, against 58% for AlexNet. Its V4, IT and behavioral scores are comparable to AlexNet's while using five convolutional and fully-connected layers instead of eight (Fig. 3). · CORnet-Z; CORnet-Z is a lightweight AlexNet alternative

  3. ShownThe fixed model-to-brain mapping partly fails for the two simpler models: in CORnet-Z and CORnet-R, the model's V2 area predicts V4 recordings better than the area designated as V4.

    For CORnet-Z, the V2 area beats the V4 area on V4 data by about .01, and CORnet-R shows the same pattern. Because the mapping is set in advance, V4 is still scored with the model's V4 area, and the authors count this as a shortcoming. · Computability; CORnet-Z is a lightweight AlexNet alternative; CORnet-R is a lightweight recurrent model

  4. ShownCORnet-R, which adds within-area recurrence and unrolls it in time in biological order, slightly improves neural fit over CORnet-Z, predicts behavior better and has higher ImageNet top-1, and can produce responses over time.

    The comparison with CORnet-Z appears only in Fig. 3; the text gives no numbers. The authors attribute the ImageNet gain to an extra convolution per area. With biological unrolling, an input reaches IT after four time steps rather than immediately. · CORnet-R; CORnet-R is a lightweight recurrent model; Figure legends

  5. ClaimedFour areas appear to be enough for current brain benchmarks. A fifth area raises CORnet-Z's ImageNet accuracy, but by the authors' account more areas do not appear to significantly affect brain-related metrics.

    The Discussion states that adding an area to CORnet-Z can give a 10% increase in ImageNet top-1, while brain-related metrics do not appear significantly affected by having more areas. No data or figure is shown for this. · Are there really only four areas in the ventral…

Methods, data and reproducibility

Three ImageNet-trained networks with four areas mapped to V1, V2, V4 and IT, one repeated circuit per model: feedforward (CORnet-Z), within-area recurrent (CORnet-R), or recurrent plus bottleneck and skip connection (CORnet-S). Scored on Brain-Score (V4, IT, human choices) and on Feedforward Simplicity: the inverse of the log of the number of convolutional and fully-connected layers along the longest path.

  • ImageNet 2012, used to train for 1000-way classification on 224 x 224 px crops
  • Brain-Score neural benchmark: time-averaged responses (70-170 ms) of V4 and IT sites to single objects on natural backgrounds, shown for 100 ms to passively fixating animals · non-human primate (monkeys) · n = 2760 images; 88 V4 sites and 168 IT sites
  • Brain-Score behavioral benchmark: two-choice object identification after a 100 ms presentation · human · n = 1472 humans; 240 of 2400 images, over three hundred thousand responses
Methods & tools
PyTorch 0.4.1, Brain-Score, AlexNet, ResNet-101, DenseNet-169, MobileNet v1, MobileNet v2, logistic regression decoder
Software & environment
PyTorch 0.4.1. SGD with momentum .9, batch size 256 and cross-entropy loss. Learning rates: 0.01 (Z) or 0.1 (R, S), divided by 10 every 10 epochs (Z, R) or every 20 epochs (S). No container or requirements file is described.
Assay conditions & compute
Single NVIDIA Titan X / GeForce 1080Ti GPU, or split over 2 GPUs for CORnet-S. Training ran 25 epochs for CORnet-Z and CORnet-R and 43 epochs for CORnet-S. CORnet-Z reached 48% top-1 in under 20 hours on one Titan X.
Reproducibility gaps
Code and trained weights are at the GitHub repository. The neural and behavioral benchmark data are cited to earlier studies, with no link or accession given here. Table 1 (the architectural parameters) is not reproduced in the text.
Limitations (6)
  • Brain-Score guided CORnet development, although it was never used directly in architecture search or optimization, so scoring CORnets on it is not a fully independent test. Validation on new experimental datasets is described as still in progress. · stated by the authors
  • The fixed area mapping already fails in part: in CORnet-Z and CORnet-R, the model's V2 area predicts V4 responses slightly better than the area designated as V4. · stated by the authors
  • The retina and lateral geniculate nucleus (LGN) are not modeled, there are no feedback or bypass connections between areas, and the first area of CORnet-S is cut to two convolutions because of its computational cost. · stated by the authors
  • Several design choices, such as the skip connection, follow engineering needs rather than cortical anatomy. Training uses supervised stochastic gradient descent, which is not biologically plausible, and the linear decoder is static, reading only the last IT output in the recurrent models. · stated by the authors
  • The neural benchmarks cover only V4 and IT (88 and 168 sites, one image set), plus one human behavioral dataset. The model's V1 and V2 areas are never compared with V1 or V2 recordings, and the numeric scores appear only in Fig. 3, not in the text. · noted in this summary
  • Neural responses are averaged over a 70-170 ms window and each CORnet is scored at its best time point, so the recurrent models' response dynamics are never tested against neural time courses. · noted in this summary

Read this if you need a compact, ImageNet-trained, partly recurrent network whose areas are assigned in advance to V1, V2, V4 and IT, to compare with primate ventral-stream recordings or behavior.

Citations
180

Abstract

Deep artificial neural networks with spatially repeated processing (a.k.a., deep convolutional ANNs) have been established as the best class of candidate models of visual processing in primate ventral visual processing stream. Over the past five years, these ANNs have evolved from a simple feedforward eight-layer architecture in AlexNet to extremely deep and branching NAS-Net architectures, demonstrating increasingly better object categorization performance and increasingly better explanatory power of both neural and behavioral responses. However, from the neuroscientists point of view, the relationship between such very deep architectures and the ventral visual pathway is incomplete in at least two ways. On the one hand, current state-of-the-art ANNs appear to be too complex (e.g., now over 100 levels) compared with the relatively shallow cortical hierarchy (4-8 levels), which makes it difficult to map their elements to those in the ventral visual stream and to understand what they are doing. On the other hand, current state-of-the-art ANNs appear to be not complex enough in that they lack recurrent connections and the resulting neural response dynamics that are commonplace in the ventral visual stream. Here we describe our ongoing efforts to resolve both of these issues by developing a \"CORnet\" family of deep neural network architectures. Rather than just seeking high object recognition performance (as the state-of-the-art ANNs above), we instead try to reduce the model family to its most important elements and then gradually build new ANNs with recurrent and skip connections while monitoring both performance and the match between each new CORnet model and a large body of primate brain and behavioral data. We report here that our current best ANN model derived from this approach (CORnet-S) is among the top models on Brain-Score, a composite benchmark for comparing models to the brain, but is simpler than other deep ANNs in terms of the number of convolutions performed along the longest path of information processing in the model. All CORnet models are available at github.com/dicarlolab/CORnet, and we plan to up-date this manuscript and the available models in this family as they are produced.

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISSchrimpf '20Nayebi '22Seibert '16Rajalingham '18Maniquet '25Du '24
77% semantic matchNeuroscience
Martin Schrimpf, Jonas Kubilius, Ha Hong+10 more·bioRxiv·Jan 2, 2020· 461 citations