henhaobio
Sign in

bioRxivBioinformaticsPreprint

AnnoAudit: a marker-based protocol for auditing single-cell atlas annotations reveals systematic, state-dependent annotation failure in a widely used traumatic brain injury resource

Li Zhang, Qi Yan, Haicheng Rao, Mengge LiiD, Xiaobo Qian, Yingshu Zhang, Rong Gao

PDFDOICodeData
In-depth full-text digest available · Jump to digest ↓

Abstract

Motivation. Single-cell atlas annotations are routinely used as ground truth without validation. We present AnnoAudit, a marker-based protocol with four convergent checks and an ambient-aware discrimination step (D1) that audits labels before re-analysis. Results. We applied AnnoAudit to labels generated by our own labelling pipeline for a public TBI single-cell dataset (CEREBRI, GSE269748). The audit exposed a permissive fallback rule that assigns a glutamatergic identity to any pan-neuronal-marker-positive cell lacking a subtype marker, manufacturing a 45,051-cell population that four independent reference-mapping models assign predominantly to choroid plexus (46.2%), astrocytes (22.4%) and microglia/macrophage (10.7%), with only 1.2% receiving any neuronal assignment (all inhibitory) and none assigned to an excitatory class. Marker- and depth-based controls show that the count matrices we analysed contain no neuronal-level expression of canonical neuronal genes (maximum Snap25 count 14 across 342,857 cells; Slc17a7 >= 5 in 5 cells), with transthyretin accounting for ~4% of all UMIs. The case illustrates the failure mode the protocol is designed to detect: permissive labelling rules can manufacture plausible-looking neuronal populations from data that lack neuronal-level signal. A simulation benchmark delimits the protocol's ambient boundary, and a literature screen identified six published re-analyses of the same dataset, none of which validated the annotations they inherited. No claim is made in this version about the cell-type annotations of any other resource. Availability and implementation. AnnoAudit is implemented in Python (scanpy); MIT-licensed source at github.com/ronnygao/AnnoAudit (doi:10.5281/zenodo.22274752). Contact. gaorong1234@suda.edu.cn

In-depth summary

Claude Opus 5.5 · xhigh effort · 8 Nov 2026 · from the full text of version 3 (28 Sept 2026) · CC BY 4.0

A marker audit finds 97.9% of CEREBRI's 45,051 official 'glutamatergic neuron' cells are not glutamatergic (four-check composite 82.6%), creating false ion-channel and OXPHOS signals; true neurons show acute KCNC3 up-regulation.

Cell-type labels from single-cell atlases are routinely reused for differential expression, trajectory and communication analyses without independent checks. The authors contrast AnnoAudit with Annotatability, scDiagnostics and scTypeEval (without benchmarking them) and present it as a lightweight audit that an individual user can run from the deposited count matrix; it also measures how contamination creates false temporal and functional signals and traces them to the cell type they came from.

Key results

  1. ShownIn mouse CEREBRI, only 963 of the 45,051 cells (2.1%) carrying the official 'glutamatergic neuron' label are confirmed by a 32-marker panel. Most of the rest are microglia, astrocytes and oligodendrocytes.

    Re-annotating 340,185 cells finds 1,174 true glutamatergic neurons; the official subset is 31.4% microglia, 18.6% astrocytes, 15.9% oligodendrocytes, 11.2% pericytes, 9.2% endothelial. Other checks agree: KMeans on 120,000 cells, 2.4% neuronal; leave-one-marker-out shift ≤1.7 pp; module scoring 96.2% non-neuronal; three CellTypist models 91.1–95.5% non-neuronal. · R1. The official “glutamatergic neuron”…; Marker-based cell-type annotation; Figure legends

  2. ShownIn mouse CEREBRI, the contamination produces a false biphasic ion-channel program and a false OXPHOS signature. Both disappear when only the marker-defined glutamatergic cells are analysed.

    Official: 40 of 307 channels (13.0%) biphasic, median 6-mo detection change −54.4%. True: zero under the primary criterion; same genes median +2.9%. GSE277487: 0/40 biphasic in 91,739 excitatory neurons. OXPHOS (KEGG mmu00190) enriched in official KCNC3+ cells (102/106 genes, FDR < 0.001, n = 2,416) but not in true ones (fold 1.24 → 1.06, p = 0.11, n = 43). · R2. Contamination generates false biphasic…; Biphasic classification

  3. ShownA trajectory-correlation fingerprint traces most of the official label's dynamics to non-neuronal cells. Across mouse CEREBRI conditions, the official per-gene trajectories follow marker-defined microglia much more closely than the true glutamatergic trajectories do.

    Over 6 complete-case conditions, official vs microglia: Gfap r = 0.94 (p = 0.005), Slc17a7 r = 0.94, Tyrobp r = 0.83 (p = 0.04). Of 244 informative ion-channel genes, 235 (96.3%) best match a non-neuronal type and 9 (3.7%) match excitatory neurons; 32 of 33 families peak with a non-neuronal type. Median r with microglia: 0.657 (official) vs 0.334 (true), Wilcoxon p = 6.9 × 10⁻¹⁸. · R2. Contamination generates false biphasic…; Signal-migration analysis; Figure legends

  4. SuggestedAfter correction, KCNC3 (Kv3.3) rises after injury in marker-defined excitatory neurons in three further TBI datasets (6 h to 21 d), the opposite of the 7-d decrease seen under the official label of mouse CEREBRI. None of these datasets covers the 6-mo window, where the official label also showed a decrease.

    GSE277487 (CCI): mean 0.33 → 0.69 at 6 h (+110%) and +40% at 2–4 d; pseudobulk +63% and +37% (p = 0.20 and 0.40; n = 2 controls). GSE160763 (fluid percussion): +35% at 7 d. GSE247339 (mild TBI): +55% at 24 h, +53% at 7 d, +68% at 21 d. Official CEREBRI label: −46% at 7 d, −45% at 6 mo. Kcnc1: +96% at 6 h in GSE277487. The text does not state the species of these three datasets. · R3. The true glutamatergic TBI response: KCNC3…; Expression metrics and statistics; Cross-dataset validation of KCNC3; Figure legends

  5. SuggestedThe same audit fails the official neuron label of a human ALS spinal cord atlas. In simulated mixtures, the composite Annotation Contamination Score (ACS) rises roughly linearly with contamination up to 20%.

    Human GSE330130 (2,450 official neurons): C1 73.3%, C2 37.8%, C3 100%, C4 N/A, ACS 70.4%. Mouse CEREBRI: C1 97.6%, C2 50.3%, C3 100%, ACS 82.6%. Simulation (c = 0–80%, 5 replicates): ACS ≈ 0.9c over 0–20% (r² = 0.999), with C2 saturating at 57% when c = 80%. The margin's AUROC is 0.996 (CEREBRI) and 1.000 (GSE330130). The failure threshold is ACS > 25%. · R4. The problem is not specific to CEREBRI…; R5. Composite Annotation Contamination Score; Composite Annotation Contamination Score (ACS); Simulation benchmark

Methods, data and reproducibility

All 340,185 CEREBRI cells were re-annotated by top score over a 32-gene, eight-type marker panel. The official label was audited by four checks (marker max-score, margin-gated module score, KMeans, gated CellTypist) averaged into an ACS calibrated on simulated mixtures; trajectory fingerprints traced ion-channel signals to source cell types, and KCNC3 was re-tested in three other TBI datasets.

  • CEREBRI cortical scRNA-seq atlas across TBI injury models and time points (official 'glutamatergic neuron' subset: 45,051 cells) · mouse · n = 340,185 cells post-QC · GSE269748
  • CCI snRNA-seq (control, 6 h, 2 d, 4 d); 91,739 marker-defined excitatory neurons · n = 162,892 cells · GSE277487
  • Fluid-percussion injury scRNA-seq (naive baseline, 7 d); 940 marker-defined excitatory neurons · n = 20,642 cells · GSE160763
  • Mild TBI scRNA-seq, hippocampus and prefrontal cortex (sham, 24 h, 7 d, 21 d); 3,295 excitatory neurons · n = 85,431 cells · GSE247339
  • ALS lumbar spinal cord snRNA-seq atlas (official neuron label: 2,450 cells) · human · n = 121,881 nuclei · GSE330130
  • Ion-channel gene set compiled from HGNC/IUPHAR nomenclature (22 families, 33 subfamily groups) with mouse ortholog matching · mouse · n = 307 genes
  • Synthetic official labels mixing marker-defined CEREBRI excitatory and microglial cells at known contamination (0, 5, 10, 20, 50, 80%) · mouse · n = 5 replicates per level
  • Works citing CEREBRI as a data or methods reference, screened for independent annotation validation (CrossRef and Google Scholar, August 2026) · n = 65 citing works
Methods & tools
AnnoAudit, scanpy, CellTypist, KMeans clustering, KEGG pathway mmu00190 (oxidative phosphorylation), HGNC/IUPHAR nomenclature, Mann–Whitney U test with Benjamini–Hochberg correction, Spearman correlation, Wilcoxon signed-rank test, CrossRef, Google Scholar
Software & environment
Python (scanpy), MIT license. The text states no package versions, container or requirements file.
Experimental conditions
The authors state the protocol runs in minutes on a laptop; no hardware or timing details are given.
Reproducibility gaps
The text does not name the CellTypist models or give package versions. The 32-marker gene list is not printed in the text; it is said to be in the repository.
Limitations (6)
  • CEREBRI contains only 1,174 marker-defined glutamatergic cells (43 KCNC3+; 13–65 per acute-window condition). This limits the power of the biphasic screen, so the cross-dataset replication carries the biological weight. · stated by the authors
  • Marker scoring can miss low-abundance transcripts in snRNA-seq data and can misassign cells in transitional states. · stated by the authors
  • Per-animal pseudobulk tests of KCNC3 in GSE277487 had only n = 2 controls and were not significant (p = 0.14–0.40), so cell-level p values should be read with this caveat. · stated by the authors
  • Only a few atlases were audited, so how common annotation contamination is across the wider atlas ecosystem remains unknown. · stated by the authors
  • C3 clusters are mapped to cell types by C1 majority, and the C2 margin's AUROC is computed against C1 labels. The checks therefore depend on the authors' own 32-marker panel more than the 'independent estimators' framing implies. · noted in this summary
  • C4 never contributed to the ACS: every CellTypist model failed the ≥80% applicability gate in both atlases, although R1 cites three CellTypist models (91.1–95.5% non-neuronal) as support. Thresholds were calibrated only on simulated CEREBRI excitatory/microglia mixtures. · noted in this summary

Read this if you reuse cell-type labels from a public single-cell atlas (CEREBRI's glutamatergic subset in particular) or want a quick marker-based check of official annotations before re-analysis.

Citations
0

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISAcri '26Qian '24Joshy '26Chaudhary '21Chi '23Qi '26
58% semantic matchBioinformatics
Dominic J. Acri, Richard Mustaklem, Liam Horan-Portelance+8 more·bioRxiv·Aug 4, 2026