A dual-metric framework for quantifying the biological fidelity of scRNA-seq pipelines
Abstract
Single-cell RNA sequencing (scRNA-seq) enables high-resolution transcriptional profiling, but early-stage processing pipelines differ markedly in barcode recovery, UMI correction, and read assignment-variations that can propagate and bias downstream analyses. We present a reproducible, parameter-aware benchmarking framework to quantify the biological fidelity of four primary pipelines-STARsolo, Cell Ranger, Kallisto|Bustools, and Alevin-fry-across simulated ground-truth datasets and complex biological contexts, including a Huntington's disease (HD) mouse model. Our approach introduces two complementary metrics: the Cluster Annotation Score (CAS), assessing concordance between direct cell-level and cluster-level consensus labels, and the Marker Concordance Score (MCS), measuring cohesion of de novo marker genes per cell type. By systematically varying highly variable gene (n_HVG) and principal component (n_PC) settings, we map how upstream quantification interacts with downstream parameter choice. Simulations show STARsolo and Kallisto|Bustools deliver high technical accuracy, but only STARsolo consistently preserves stable cell identities and coherent marker expression across parameter regimes. In empirical datasets, alignment-based pipelines (STARsolo, Cell Ranger) yield higher CAS/MCS values and more biologically faithful annotations, while alignment-free methods show reduced signal fidelity despite faster runtimes. Differences introduced during primary processing persist after batch correction and integration, altering disease-associated cell type detection. Our open-source CAS-MCS-Scoring toolkit enables transparent evaluation of pipeline performance, providing a practical guide for selecting analysis strategies that maximize reproducibility and biological interpretability in scRNA-seq studies.
02 /Specification & Classification
Click any tag to browse matching papers- Citations
- 0
- Organism
- Assay
- Research Focus
- Approach
Reproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- scPipe: a flexible R/Bioconductor preprocessing pipeline for single-cell RNA-sequencing data
Luyi Tian, Shian Su, … Matthew E. Ritchie·PLoS Computational BiologyPeer-reviewed·Aug 10, 2018
- Comparative Analysis of common alignment tools for single cell RNA sequencing
Ralf Schulze Brüning, Lukas Stefan Tombor, … David John·GigaSciencePeer-reviewed·Jan 1, 2022
- Multi-level cellular and functional annotation of single-cell transcriptomes
Nicholas Mikolajewicz, Kevin R. Brown, … Hong Ryul Han·bioRxiv·Mar 13, 2022
- A Comprehensive Multi-Center Cross-platform Benchmarking Study of Single-cell RNA Sequencing Using Reference Samples
Wanqiu Chen, Yongmei Zhao, … Charles Wang·Nature BiotechnologyPeer-reviewed·Dec 21, 2020
- pipeComp, a general framework for the evaluation of computational pipelines, reveals performant single-cell RNA-seq preprocessing tools
Pierre‐Luc Germain, Anthony Sonrel, Mark D. Robinson·Genome biologyPeer-reviewed·Sep 1, 2020
- Beyond benchmarking: towards predictive models of dataset-specific single-cell RNA-seq pipeline performance
Cindy Fang, Alina Selega, Kieran R. Campbell·Genome biologyPeer-reviewed·Jun 17, 2024