Microbiome differential abundance methods produce disturbingly different results across 38 datasets
Jacob T. NearingiD, Gavin M. DouglasiD, Molly G. HayesiD, Jocelyn MacDonald, Dhwani DesaiiD, Nicole E. McCormick, Casey M. A. JonesiD, Robyn WrightiD, Akhilesh S. DhananiiD, A. ComeauiD, Morgan G. I. LangilleiD
Abstract
Identifying differentially abundant microbes is a common goal of microbiome studies. Multiple methods have been applied for this purpose, which are largely used interchangeably in the literature. Although it has been observed that these tools can produce different results, there have been very few large-scale comparisons to describe the scale and significance of these differences. In addition, it is challenging for microbiome researchers to know which differential abundance tools are appropriate for their study and how these tools compare to one another. Here, we have investigated these questions by analyzing 38 16S rRNA gene datasets with two sample groups for differential abundance testing. We tested for differences in amplicon sequence variants and operational taxonomic units (referred to as ASVs for simplicity) between these groups with 14 commonly used differential abundance tools. Our findings confirmed that these tools identified drastically different numbers and sets of significant ASVs, however, for many tools the number of features identified correlated with aspects of the tested study data, such as sample size, sequencing depth, and effect size of community differences. We also found that the ASVs identified by each method were dependent on whether the abundance tables were prevalence-filtered before testing. ALDEx2 and ANCOM produced the most consistent results across studies and agreed best with the intersect of results from different approaches. In contrast, several methods, such as LEfSe, limma voom, and edgeR, produced inconsistent results and in some cases were unable to control the false discovery rate. In addition to these observations, we were unable to find supporting evidence for a recent recommendation that limma voom, corncob, and DESeq2 are more reliable overall compared with other methods. Although ALDEx2 and ANCOM are two promising conservative methods, we argue that those researchers requiring more sensitive methods should use a consensus approach based on multiple differential abundance methods to help ensure robust biological interpretations.
02 /Specification & Classification
Click any tag to browse matching papers- Citations
- 1,005
- FWCI
- 63.05
- Field percentile
- top 1%
- Organism
- Assay
- Research Focus
- Approach
In-depth summary
Claude Opus 5.5 · xhigh effort · 8 Nov 2026 · from the full text of version 1 (10 May 2021) · CC BY 4.0
In 38 two-group 16S rRNA datasets, 14 differential abundance tools disagreed sharply; ALDEx2 and ANCOM-II were conservative and consistent, while edgeR and LEfSe inflated false positives.
Microbiome studies use differential abundance (DA) tools interchangeably, yet earlier evaluations disagree on false discovery control, lack gold standards, omit tools such as MaAsLin2 and use few datasets. This independent test on real 16S data measures how tool choice and prevalence filtering alter significant taxa, and checks a prior recommendation favoring limma voom, corncob and DESeq2.
Key results
ShownAcross 38 datasets, the share of ASVs called significant differed widely by tool, and tools ranked inconsistently between datasets. limma voom, Wilcoxon (CLR), LEfSe and edgeR were permissive, while ALDEx2 and ANCOM-II were consistently the most conservative.
Per-tool mean significant ASVs spanned 0.8-40.5% unfiltered and 3.8-32.5% after 10% prevalence filtering. Unfiltered means: limma voom (TMMwsp) 40.5%, Wilcoxon (CLR) 30.7%, LEfSe 12.6%, edgeR 12.4%, ALDEx2 1.4%, ANCOM-II 0.8%. In Human-HIV (3), limma voom (TMMwsp) called 73.5% vs 0-11% for other tools. · Microbiome differential abundance methods…
ShownWhen case/control labels were assigned at random (no true signal), most tools called few ASVs. edgeR and LEfSe stood out on filtered data and limma voom on unfiltered data; only ALDEx2 and the rarefied t-test never called any ASV significant.
Filtered: edgeR mean 10.3% (SD 9.0%), LEfSe 4.4% (SD 1.2%) vs 0%-2.2% for other tools. Unfiltered: both limma voom variants averaged >5% significant ASVs in 5/8 datasets; in Freshwater-Treatment, 0.001% (TMMwsp) vs 9.0% (TMM). Wilcoxon (CLR) replicates with 30% or more significant ASVs had larger between-group read depth differences. · False positive analysis; False discovery rate of microbiome differential…
ShownMost ASVs flagged by ALDEx2 and ANCOM-II were also called by almost all other tools. edgeR and LEfSe produced the most tool-unique ASVs and separated from other tools in a Jaccard-based ordination. Prevalence filtering increased agreement between tools.
ASVs called by no other tool: edgeR 12.1%, LEfSe 11.1%. Proportion of ASVs called by more than 12 tools: mean 38.5 (SD 15.8) filtered vs 17.3 (SD 22.1) unfiltered. edgeR and LEfSe separated from other methods on the first principal coordinate for both unfiltered and filtered data. · High variability of overlapping significant ASVs
SuggestedAt genus level across five diarrhea case-control datasets, every tool was more consistent across studies than a random-sampling null, with ALDEx2 highest and edgeR lowest. In five obesity datasets, only MaAsLin2 and the rarefied t-test beat chance.
Diarrhea: ALDEx2 1.35-fold over random expectation (p < 0.0001) vs edgeR 1.10-fold (p = 0.02); all tools p < 0.05. ALDEx2 averaged 17.6 significant genera per dataset (SD 17.4) vs 46.0 (SD 12.9) for edgeR. Obesity: MaAsLin2 (rare) was the most consistent (fold difference 1.23; p = 0.006). · Cross-study differential abundance consistency…; Tools vary in how consistently they identify the…
ShownHit counts tracked dataset properties. Every tool correlated positively with between-group effect size, and after filtering all tools also correlated with sample size and read depth; these correlations were less uniform without filtering.
Spearman rho vs Aitchison-distance PERMANOVA effect size: 0.35-0.72 unfiltered, 0.31-0.52 filtered. Unfiltered: only the t-test, both Wilcoxon and both limma voom variants correlated with read-depth range, and edgeR correlated negatively with mean sample richness. Filtering lowered total hits for all tools except ALDEx2. · Comparing numbers of significant hits between…; Microbiome differential abundance methods…
Methods, data and reproducibility
14 differential abundance workflows (ALDEx2, ANCOM-II, corncob, DESeq2, edgeR, LEfSe, limma voom, MaAsLin2, metagenomeSeq, t-test, Wilcoxon variants) were run on 38 two-group 16S rRNA datasets with and without 10% prevalence filtering. False discovery rate was estimated by random case/control relabeling, and genus consistency across studies was tested against a random null.
- 38 published or public two-group 16S rRNA gene datasets (ASV or OTU tables) spanning human gut, plastisphere, freshwater lake and urban environments · multiple; human-associated and environmental microbiomes · n = 38 datasets · Raw data listed in Supplementary Table 1; processed tables on figshare 14531724
- Large, environmentally diverse subset of the main datasets; the most frequent group was split by random case/control labels to estimate false discovery rate · multiple; subsets of the 38 datasets · n = eight datasets; 10 (filtered) or up to 100 (unfiltered) replicates per tool
- Diarrhea vs non-diarrhea 16S datasets (two added beyond the main set), collapsed to genus level for cross-study consistency testing · human (individuals with vs without diarrhea) · n = five datasets · Supplementary Table 1
- Obesity vs control 16S datasets processed with the same genus-level workflow (supplementary comparison) · human (obese vs control individuals) · n = five datasets
- Methods & tools
- QIIME 2 (version 2019.7) with cutadapt, VSEARCH join-pairs and Deblur, ALDEx2, ANCOM-II, corncob, DESeq2, edgeR, LEfSe, limma voom (TMM and TMMwsp), MaAsLin2 (rarefied and non-rarefied), metagenomeSeq, Welch's t-test and Wilcoxon rank-sum test (rarefied and CLR-transformed), R 3.6.3 with vegan (adonis PERMANOVA), phyloseq, pheatmap and Matching (ks.boot)
- Code
- https://github.com/nearinj/Comparison_of_DA_microbiome_methodshttps://github.com/FrederickHuangLin/ANCOMhttps://joey711.github.io/phyloseq-extensions/edgeR.html
- Software & environment
- R version 3.6.3; QIIME 2 version 2019.7 (Microbiome Helper SOP, cutadapt, VSEARCH, Deblur) for raw reads; all tools run through a custom run_all_tools.sh script.
- Reproducibility gaps
- Raw-data accessions appear only in Supplementary Table 1, not in the main text; the text does not state versions of the individual DA R packages.
Limitations (6)
- Estimating FDR by relabeling is imperfect: real groupings probably carry small systematic differences while relabeled groups carry none, so calling few hits under relabeling does not prove a low FDR in practice. · stated by the authors
- The cross-study null assumed every genus is equally likely to be significant by chance, which cannot fully hold because genera differ in prevalence across samples. · stated by the authors
- Different methods were used to classify taxa in the diarrhea datasets, adding technical variation. Obesity datasets showed chance-level consistency for most tools, possibly because of dataset-specific technical or biological factors. · stated by the authors
- The 10% prevalence cutoff was chosen arbitrarily; the authors note that more work is needed to identify an optimal filtering threshold. · stated by the authors
- No spike-in, mock-community or simulated-signal benchmark was used. The lower sensitivity and higher precision of ALDEx2 and ANCOM-II are therefore inferred from agreement between tools, not measured against ground truth. · noted in this summary
- On unfiltered data, which the authors describe as less stable, ALDEx2, ANCOM-II and corncob had only 10 relabeling replicates versus 100 for other tools, so their FDR estimates are less precise. · noted in this summary
Who should read:Read this if you pick or review differential abundance methods for 16S microbiome studies and want empirical evidence of how tool choice and prevalence filtering change which taxa come out significant.
Reproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- A configuration-resolved benchmark of differential abundance analysis methods for human gut 16S rRNA microbiome data
Nader Abdelmalek, Carola Di Meo, … Alessandro Tanca·bioRxiv·Sep 3, 2026
- A Permutation-Based Framework for Evaluating Bias in Microbiome Differential Abundance Analysis
Ke Zeng, Anthony A. Fodor·bioRxiv·May 11, 2026
- Commonly used compositional data analysis implementations are not advantageous in microbial differential abundance analyses benchmarked against biological ground truth
Samuel David Gamboa-Tuz, Marcel Ramos, … Levi Waldron·bioRxiv·Feb 13, 2025
- ConsensusMetaDA: an R Package for consensus-based differential abundance analysis of metagenomic data.
Manoharan Kumar, Matthew A. Field·bioRxiv·Nov 3, 2025
- LOCOM2: Robust Differential Abundance Analysis for Microbiome Data
Mengyu He, Glen A. Satten, Yi‐Juan Hu·bioRxiv·Apr 9, 2026
- Multi-scale Adaptive Differential Abundance Analysis in Microbial Compositional Data
Shulei Wang·BioinformaticsPeer-reviewed·Apr 1, 2023