Genome biologyBioinformaticsPeer-reviewed journal article
Commonly used software tools produce conflicting and overly-optimistic AUPRC values
Wenyu Chen, Miao CheniD, Zhenghao ZhangiD, Cathy Sin‐Hang Fung, Ran WangiD, Yizhen CheniD, Qian YaniD, Lixin ChengiD, Kevin Yuk-Lap YipiD, Stephen Kwok‐Wing TsuiiD, Qin CaoiD
- Citations
- 8
- FWCI
- 1.18
- Field percentile
- top 21%
Abstract
The precision-recall curve (PRC) and the area under it (AUPRC) are useful for quantifying classification performance. They are commonly used in situations with imbalanced classes, such as cancer diagnosis and cell type annotation. We evaluated 10 popular tools for plotting PRC and computing AUPRC, which were collectively used in >3,000 published studies. We found the AUPRC values computed by the tools rank classifiers differently and some tools produce overly-optimistic results.
Reproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- Exact Probability Distribution for the ROC Area Under Curve
Joakim Ekström, Jim Ögren, Tobias Sjöblom·CancersPeer-reviewed·Mar 15, 2023
- MLcps: Machine Learning Cumulative Performance Score for classification problems
Akshay Akshay, Masoud Abedi, … Ali Hashemi Gheinani·GigaSciencePeer-reviewed·Dec 28, 2022
- ChatGPT-Enhanced ROC Analysis (CERA): A Shiny Web Tool for Finding Optimal Cutoff in Biomarker Analysis
Melih Ağraz, Christos Socrates Mantzoros, George Em Karniadakis·PLoS ONEPeer-reviewed·Apr 10, 2024
- ARA: accurate, reliable and active histopathological image classification framework with Bayesian deep learning
Alicja Rączkowska, Marcin Możejko, … Ewa Szczurek·Scientific ReportsPeer-reviewed·Oct 4, 2019
- Feasibility of Automated Deep Learning Design for Medical Image Classification by Healthcare Professionals with Limited Coding Experience
Livia Faes, Siegfried K. Wagner, … Pearse Andrew Keane·bioRxiv·May 27, 2019
- Beyond accuracy: Measures for assessing machine learning models, pitfalls and guidelines
Richard Dinga, Brenda W.J.H. Penninx, … André F. Marquand·bioRxiv·Aug 22, 2019