Assay concordance sets exact ceilings on what one biological score can predict
Abstract
Computational models of biology are ranked by averaging one prediction against many experimental realizations of a phenotype that are treated as interchangeable. We show this imposes an exact, model-free ceiling fixed by how much those realizations agree with each other, and that the ceiling depends on the evaluation metric through a single support-function identity. Measuring assay concordance across four public registries, 2,822 MaveDB score sets, 217 ProteinGym assays, two drug screens and 1,150 CRISPR cell lines, we find that two assays of one target agree at 0.56-0.68, and that 541 domains measured twice with different proteases fix assay reliability at 0.897, so 70-90% of every ceiling is irreducible biology rather than noise. Published predictors realize 63% of the achievable on the correlation benchmarks report and 18% on the top-1% selection their users perform. We provide the estimator, the ceilings, and the measurements the field has not made.
02 /Specification & Classification
Click any tag to browse matching papers- Citations
- 0
- Assay
- Research Focus
- Approach
Reproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- Metrics for Evaluating Biological AI Model Predictive Accuracy at the Data-Substrate Level
M. Tony Ewing·bioRxiv·Jun 18, 2026
- Two Comparators May Be All We Need
Steven M. Muskal·bioRxiv·Oct 8, 2026
- Dynamic applicability domain (dAD) for compound-target binding affinity prediction with confidence guarantees
Davor Oršolić, Tomislav Šmuc·bioRxiv·Aug 22, 2022
- FAIRification of computational models in biology
Irina Balaur, David Phillip Nickerson, … Dagmar Waltemath·bioRxiv·Mar 21, 2025
- The field of protein function prediction as viewed by different domain scientists
Rashika Ramola, Iddo Friedberg, Predrag Radivojac A·Bioinformatics AdvancesPeer-reviewed·Jan 1, 2022
- Merging Bioactivity Predictions from Cell Morphology and Chemical Fingerprint Models Using Similarity to Training Data
Srijit Seal, Hongbin Yang, … Andreas Bender·Journal of CheminformaticsPeer-reviewed·Jun 2, 2023