henhaobio
Sign in

Nature MethodsGenomicsPeer-reviewed journal article

Systematic assessment of long-read RNA-seq methods for transcript identification and quantification

Francisco J. Pardo-PalaciosiD, Dingjie WangiD, Fairlie ReeseiD, Mark DiekhansiD, Sílvia Carbonell SalaiD, Brian A. WilliamsiD, Jane E. LovelandiD, Maite De MaríaiD, Matthew S. AdamsiD, Gabriela Balderrama-GutierreziD, Amit K. BeheraiD, José M. GonzáleziD, Toby HuntiD, Julien LagardeiD, Cindy LiangiD, Haoran LiiD, Marcus Jerryd MeadeiD, David A. Moraga Amador, Andrey D. PrjibelskiiD, İnanç BiroliD, Hamed BostaniD, Ashley M. BrooksiD, Muhammed Hasan ÇelikiD, Ying CheniD, Mei R. M. DuiD, , Angela Norie BrooksiD

PDFDOIPreprintCodeData
In-depth full-text digest available · Jump to digest ↓

Abstract

AbstractThe Long-read RNA-Seq Genome Annotation Assessment Project (LRGASP) Consortium was formed to evaluate the effectiveness of long-read approaches for transcriptome analysis. The consortium generated over 427 million long-read sequences from cDNA and direct RNA datasets, encompassing human, mouse, and manatee species, using different protocols and sequencing platforms. These data were utilized by developers to address challenges in transcript isoform detection and quantification, as well as de novo transcript isoform identification. The study revealed that libraries with longer, more accurate sequences produce more accurate transcripts than those with increased read depth, whereas greater read depth improved quantification accuracy. In well-annotated genomes, tools based on reference sequences demonstrated the best performance. When aiming to detect rare and novel transcripts or when using reference-free approaches, incorporating additional orthogonal data and replicate samples are advised. This collaborative study offers a benchmark for current practices and provides direction for future method development in transcriptome analysis.

In-depth summary

Claude Opus 5.5 · xhigh effort · 8 Nov 2026 · from the full text of version 1 (27 Jul 2023) · CC BY 4.0

Benchmarking six long-read RNA-seq protocol-platform combinations and 14 tools across human, mouse, and manatee shows read length and accuracy drive isoform detection, depth drives quantification.

Long-read RNA-seq is increasingly used to identify and quantify isoforms, yet the effects of platform, library protocol, and analysis software needed systematic comparison; the earlier short-read RGASP effort found limited recall of full-length transcripts. LRGASP provides shared replicated data, developer-submitted predictions, several ground truths, and PCR validation to inform method choice.

Key results

  1. ShownThe analysis tool, more than sequencing depth, set how many genes and transcripts were reported. Pipelines agreed poorly except on annotated full-splice-match (FSM) isoforms, and longer, more accurate reads (cDNA-PacBio, R2C2-ONT) favored detection.

    WTC11 submissions reported 399-23,647 known genes and 524-329,131 transcripts. CapTrap-ONT and cDNA-ONT gave roughly ten times more reads, but read number was not associated with transcripts detected. Transcripts seen in at least three datasets and three tools numbered around 45,000 unique intron chains (UICs) per sample, about 8% of all UICs, with about 80% FSM. · LRGASP Data and study design; Challenge 1 Results and Evaluation: Transcript…; Discussion

  2. ShownShort-read RSEM was still the most accurate quantifier, and IsoQuant, IsoTools, and FLAIR led the long-read tools. Higher-depth ONT cDNA and CapTrap data gave the most reproducible estimates, and annotation accuracy strongly affected quantification error.

    Cell mixing: RSEM Spearman correlation (SCC) 0.87 and median relative difference 0.13, vs IsoQuant on cDNA-ONT 0.85 and 0.14; Bambu SCC 0.53. Mean irreproducibility: CapTrap-ONT 0.19, cDNA-ONT 0.20, dRNA-ONT 0.66. Simulations: RSEM with an inaccurate annotation had mean normalized RMSE 1.70, vs 0.62 for long-read tools and 0.52 with the accurate annotation. · Challenge 2 Results and Evaluation: Transcript…; Discussion; Final recommendations

  3. ShownTargeted RT-PCR followed by long-read sequencing of the amplicons confirmed many novel WTC11 isoforms, including some reported by only one or two pipelines. Validation success tracked expression and the number of datasets supporting an isoform.

    Primers covered 178 WTC11 target regions. Validation rates: GENCODE-known 81% (n=26), GENCODE-novel 63% (n=41), GENCODE-rejected 22% (n=9); consistently detected novel isoforms 100%; rarely detected (1 or 2 pipelines) novel-in-catalog 91% and novel-not-in-catalog 57%. Overall, 743 novel isoforms were detected consistently vs 242,125 rarely. · Experimental validation of transcript isoform…; Discussion

  4. ShownThe ground-truth benchmarks disagreed. Spike-ins implied high accuracy for most tools, but simulations and expert manual annotation showed that novel isoforms were recovered much less well than annotated ones.

    Spliced SIRVs: most tools had high sensitivity, except TALON. On 50 GENCODE manually annotated human loci (271 accepted WTC11 models, mostly novel), sensitivity for annotator-defined novel transcripts was close to 0 for most pipelines; only Iso_IB exceeded 0.4. Simulated data gave higher novel sensitivity than this manual set. · Challenge 1 Results and Evaluation: Transcript…; Discussion

  5. ShownWithout a reference annotation, de novo isoform identification varied widely. Bambu and StringTie2+IsoQuant showed low sensitivity with moderate to good precision, RNA-Bloom was sensitive but imprecise, and rnaSPAdes produced many short, fragmented models.

    Predictions ranged from about 20K to 150K transcripts in mouse ES and 2K to 500K in manatee. Mouse ES SIRV sensitivity was about 70% for RNA-Bloom vs about 25% for StringTie2+IsoQuant and Bambu. rnaSPAdes had about 40% non-canonical junctions, and Bambu recovered no SIRVs in manatee. · Challenge 3 Results and Evaluation: De novo…

Methods, data and reproducibility

Organizers sequenced replicated human and mouse cell-line RNA (with SIRV spike-ins) and manatee blood using cDNA, CapTrap, R2C2, and direct RNA libraries on PacBio and Oxford Nanopore (ONT). Tool developers submitted predictions, which independent evaluators scored with SQANTI3, nine quantification metrics, simulations, manual annotation, and PCR validation.

  • LRGASP RNA-seq corpus: cDNA, CapTrap, R2C2, and direct RNA libraries on PacBio and ONT plus Illumina, with Lexogen SIRV-Set 4 spike-ins, deposited at the ENCODE DCC · human, mouse, manatee · n = over 427 million long reads · syn25683364 (GRCh38 reference with SIRVs and EBV given to participants)
  • Human WTC11 induced pluripotent stem cell line (Challenges 1 and 2), with matched Illumina, CAGE, and Quant-seq data · human · n = biological triplicates
  • Mouse 129/Casteneus embryonic stem cell line (Challenge 1; also annotation-free reconstruction in Challenge 3) · mouse · n = biological triplicates
  • H1-hESC, H1-derived definitive endoderm, and their undisclosed-ratio mix (H1-mix) for quantification (Challenge 2) · human · n = biological triplicates
  • Florida manatee whole blood, one pooled sample with excess SIRV spike-ins, plus a long-read draft genome assembled for Challenge 3 · Floridian manatee (Trichechus matatus) · n = single pool of nine individual RNA samples · JARVKP000000000.1; GCF_000243295.1 (Illumina draft genome)
  • Simulated reads (NanoSim for ONT, IsoSeqSim for PacBio, RSEM for Illumina) including novel transcripts hidden from participants · human and mouse · n = six simulation datasets (Challenge 2)
  • Exhaustive GENCODE manual annotation of undisclosed loci, done independently per library preparation, as real-data ground truth · human and mouse · n = 50 human and 50 mouse loci; 271 accepted WTC11 models
  • Targeted RT-PCR amplicons from WTC11 and manatee RNA, sequenced on ONT (R10.4.1) and PacBio Sequel II · human; Floridian manatee · n = 178 WTC11 target regions; 26 manatee transcript predictions · SRR23881262; SRR24680098; SRR24680099
Methods & tools
SQANTI3, minimap2 (2.24-r1122), Bambu, IsoQuant, FLAIR, FLAMES, IsoTools, TALON-LAPA (isoform detection and quantification), Mandalorion, LyRic, Spectra, Iso_IB, StringTie2 (isoform detection), RNA-Bloom, rnaSPAdes, StringTie2+IsoQuant (annotation-free isoform identification), RSEM (short-read quantification control), NanoSim, IsoSeqSim (long-read simulators), BUSCO (eutheria_odb10), Primers-JuJu with Primer3 and In-Silico PCR (isoform-specific primer design), CapTrap and R2C2 library protocols, Flye and Pilon (manatee genome assembly and polishing), Lexogen SIRV-Set 4 spike-ins
Reagents & environment
minimap2 2.24-r1122; guppy 6.2.11 high-accuracy model (dna_r10.4.1_e8.2_260bps_hac.cfg); GENCODE v39 (human) and vM28 (mouse) for evaluation; BUSCO lineage eutheria_odb10.
Reproducibility gaps
Access to data, code, and evaluation scripts is described only in the Supplementary Information. The ENCODE DCC read deposits have no accessions in the text, and the OpenEBench benchmarking resource is named without a URL.
Limitations (6)
  • Long-read alignment was not benchmarked. All tools and SQANTI3 relied on minimap2, and manual curation found mapping errors that produced wrong junction models; in ALG6 an annotated U12 GT-AT intron was forced into a GT-AG intron, so no pipeline recovered that isoform. · stated by the authors
  • Participants submitted one self-configured prediction set per case, although most tools are highly tunable, so rankings may not reflect performance under other settings or analysis goals. · stated by the authors
  • Participation was unbalanced because submitters picked which data types to analyze (e.g., the tools reporting the most transcripts processed only PacBio data), which may have influenced some conclusions. · stated by the authors
  • Several long-read tools (TAMA, IsoSeq3, LIQA, ESPRESSO) did not take part, and continuing gains in sequencing throughput and accuracy may require revising some conclusions. · stated by the authors
  • Each ground truth is imperfect. SIRVs are low-complexity and known to submitters, simulators model sequencing errors but not library or biological artifacts, and GENCODE-curated loci were biased toward fewer reads and replicates. · stated by the authors
  • Rare-isoform validation rates come from a small, manually selected target set that favored isoforms above 10 TPM where possible, and success tracked abundance. Extrapolating these rates to the 242,125 rarely detected WTC11 isoforms is therefore likely optimistic. · noted in this summary

Read this if you are choosing a long-read RNA-seq platform, library protocol, or isoform detection or quantification tool, or need community ground truths and metrics to benchmark a new method.

Citations
235
FWCI
29.59
Field percentile
top 1%

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISYu '26Ji '25Calvo-Roitb '24Hoffmann '25Jetzinger '25Chen '25
71% semantic matchBioinformatics
Houlin Yu, Christophe H. Georgescu, Akanksha Khorgade+20 more·bioRxiv·Feb 12, 2026· 2 citations