Variant effect predictors are shaping the ClinVar classifications used to train and evaluate them
Abstract
Variant effect predictors (VEPs) are playing an increasingly influential role in clinical variant classification. Following recent calibration efforts and the development of points-based classification standards, computational predictions can now carry enough weight to move variants out of uncertain significance with little additional evidence. While clinically valuable, this increases the potential for classification-level circularity, where VEP predictions help determine the clinical classifications later used to train, benchmark and calibrate VEPs. Here, we show that computational evidence is already leaving a detectable signature in ClinVar. Among rare missense variants from a major submitter, likely benign classifications shifted markedly towards more benign scores and show a sharp step change at the calibrated moderate-evidence threshold of BayesDel_noAF. This signal extends to nearly all 52 predictors examined, including population-free models unlikely to have been used in classification, probably reflecting shared evolutionary information. Classifications informed by computational evidence therefore cannot be treated as independent ground truth, and temporal holdouts cannot eliminate this circularity. To prevent a self-reinforcing feedback loop between prediction and classification, we recommend that ClinVar submissions routinely record the evidence criteria and computational tools supporting each classification in a standardised format, alongside greater use of classification-independent benchmarks.
02 /Specification & Classification
Click any tag to browse matching papers- Citations
- 0
- Organism
- Assay
- Research Focus
- Approach
Reproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- Calibration of additional computational tools expands ClinGen recommendation options for variant classification with PP3/BP4 criteria
Timothy R. Bergquist, Sarah L. Stenton, … Tina F. Pesaran·Genetics in MedicinePeer-reviewed·Mar 10, 2025
- Gene- and domain-aware calibration increases the clinical utility of variant effect predictors
Y P Chen, Shawn Fayer, … Vikas Rao Pejaver·bioRxiv·Mar 31, 2026
- Evaluating the impact of in silico predictors on clinical variant classification
Emma H. Wilcox, Mahdi Sarmady, … Ahmad N. Abou Tayoun·Genetics in MedicinePeer-reviewed·Dec 23, 2021
- Clinical evidence yield as a framework for evaluating computational predictors and multiplexed assays of variant effect
Yifei Shang, Mihaly Badonyi, Joseph A. Marsh·bioRxiv·Mar 30, 2026
- Variant effect predictor correlation with functional assays is reflective of clinical classification performance
Benjamin Livesey, Joseph A. Marsh·Genome biologyPeer-reviewed·Apr 22, 2025
- Language models reveal evidence gaps in variants of uncertain significance
Weijiang Li, Vineel Bhat, … Christopher A. Cassa·medRxiv·Mar 2, 2026