Clinical evidence yield as a framework for evaluating computational predictors and multiplexed assays of variant effect
Abstract
Interpreting the clinical significance of missense variants of uncertain significance (VUS) remains a major challenge in clinical genetics. Although computational variant effect predictors (VEPs) and multiplexed assays of variant effect (MAVEs) can generate large-scale functional scores, their value is typically assessed using discrimination metrics such as AUROC rather than by the strength of evidence they provide under ACMG/AMP guidelines. Here, we introduce mean evidence strength (MES), a quantitative metric that summarises the pathogenic and benign evidence assigned across missense variants following gene-level Bayesian calibration. Using the acmgscaler framework, we calibrated 12 population-free VEPs across 367 disease genes and analysed 15 MAVE datasets with sufficient clinical data. MES revealed important discrepancies with AUROC, including cases where methods with similar discrimination differed substantially in evidence yield. MAVEs achieved high average MES despite lower AUROC, while several VEPs showed strong discrimination but more limited calibrated evidence. Among predictors, CPT-1 achieved the highest MES and provided moderate or stronger evidence for the largest fraction of ClinVar VUS. MES therefore provides a practical framework for evaluating computational and experimental variant effect datasets in terms of calibrated clinical evidence yield.
02 /Specification & Classification
Click any tag to browse matching papers- Citations
- 1
- Organism
- Assay
- Research Focus
- Approach
Reproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- Gene- and domain-aware calibration increases the clinical utility of variant effect predictors
Y P Chen, Shawn Fayer, … Vikas Rao Pejaver·bioRxiv·Mar 31, 2026
- Why variant effect predictors and multiplexed assays agree and disagree
Benjamin Livesey, Joseph A. Marsh·Nature CommunicationsPeer-reviewed·Aug 27, 2026
- Variant effect predictors are shaping the ClinVar classifications used to train and evaluate them
Minyu Tao, Joseph A. Marsh·bioRxiv·Oct 27, 2026
- Combining MAVEs and computational predictors improves variant classification across ancestries in hereditary cancer genes
Nikita Bora, Mihaly Badonyi, … Joanne Yuen Yie Ngeow·medRxiv·Dec 9, 2025
- Beyond a single threshold: calibration of CADD v1.7 and AlphaGenome for ACMG/AMP computational evidence across variant classes, model versions, and reference data
José Patricio Miranda, Lorena Carvajal, … Cristian Saavedra·bioRxiv·Nov 2, 2026
- Disagreement among variant effect predictors guides experimental prioritization of target proteins
Nicolas Jonsson, Joseph A. Marsh, Kresten Lindorff‐Larsen·bioRxiv·Mar 20, 2026