Protein language models are biased by unequal sequence sampling across the tree of life
Abstract
Protein language models (pLMs) trained on large protein sequence databases have been used to understand disease and design novel proteins. In design tasks, the likelihood of a protein sequence under a pLM is often used as a proxy for protein fitness, so it is critical to understand what signals likelihoods capture. In this work we find that pLM likelihoods unintentionally encode a species bias: likelihoods of protein sequences from certain species are systematically higher, independent of the protein in question. We quantify this bias and show that it arises in large part because of unequal species representation in popular protein sequence databases. We further show that the bias can be detrimental for some protein design applications, such as enhancing thermostability. These results highlight the importance of understanding and curating pLM training data to mitigate biases and improve protein design capabilities in under-explored parts of sequence space.
02 /Specification & Classification
Click any tag to browse matching papers- Citations
- 57
- Organism
- Assay
- Research Focus
- Approach
In-depth summary
Claude Sonnet 4.6 · xhigh effort · 8 Nov 2026 · from the full text of version 1 (12 Mar 2024) · CC BY 4.0
Likelihoods from Progen2 and ESM2 protein language models are systematically higher for sequences from some species, tracking evolution-weighted database sequence counts, and pull likelihood-guided designs away from extremophile properties.
Protein language model likelihoods are often used as a proxy for protein fitness in design, so it matters what signals they capture. The paper reports a species bias tied to uneven sequence sampling in UniProt, and contrasts it with a previously described sequence similarity bias.
Key results
ShownSpecies identity alone explains much of the variation in pLM likelihoods across orthologous proteins, and Elo ratings of species differ widely in both Progen2 and ESM2.
Dataset: 133 species, 203 protein types, 7545 sequences. For Progen2-xlarge, species explains 50% of variance alone and 67% after controlling for protein type. Elo gap between 25th and 75th percentile species: 510 (95% win chance) for Progen2-xlarge, 220 (80%) for ESM2-15B. Elo Pearson correlation between the two models: 0.83. · 4 pLM likelihoods are higher for sequences from…; 4.1 Results
SuggestedThe bias is largely explained by species representation in databases once evolutionary closeness is accounted for. Raw per-species counts explain little; evolution-weighted counts explain much more.
Spearman correlation of Elo with raw Swiss-Prot counts is below 0.3 (most species other than H. sapiens, M. musculus and E. coli are poorly fit). With evolution-weighted counts it is 0.73 (Progen2-xlarge) and 0.606 (ESM2-15B). Appendix C reports the same direction for UniRef90 and two-tiered sampling counts. The evidence is correlational. · 5 pLM bias is largely explained by species…; C Robustness to different database sequence…
ShownIn simulated likelihood-guided design with Progen2-xlarge, designs drift towards sequences similar to orthologs from high-Elo species, most strongly when the starting sequence comes from a low-Elo species.
1092 designs in total (Gibbs sampling, 10,000 steps, lambda = 1, 3 seeds per start). For 20 proteins with 15-20 species' orthologs as starts, similarity-weighted Elo increases after design (paired t-test t = 15.2, p = 7.7e-47). · 6.1 Overview of simulated design; 6.2 Protein design has a species drift; D.1 Details for similarity-weighted Elo
SuggestedDesign starting from thermophilic species' orthologs tends to lower predicted thermostability.
13 proteins with 3-7 thermophilic orthologs each (7 thermophilic species). DeepSTABp predicted Tm falls after design (t = -7.1, p = 3.5e-11). 63% of designs have lower predicted Tm, and in one-third of those the drop exceeds 20°C. Predictions only, no experimental assay. · 6.3 Protein design diminishes extremophile…; D.2 Details for thermostability
SuggestedDesign starting from halophilic species' orthologs tends to raise isoelectric point, which the authors read as lost salt tolerance. The test set is ribosomal proteins from 3 halophilic species.
18 ribosomal proteins with 1-3 halophilic orthologs each. Calculated pI rises after design (t = 4.6, p = 2.0e-5). 83% of designs have higher pI, with an average increase of 4.6. A number of final design sequences (in the thermostability and salt analyses) have >90% identity to mammalian orthologs. · 6.3 Protein design diminishes extremophile…; D.3 Details for salt tolerance; D.4 Convergence to high Elo orthologs after…
Methods, data and reproducibility
The authors collected Swiss-Prot orthologs across 133 species, scored them with Progen2 and ESM2, and ranked species by Elo ratings built from pairwise likelihood comparisons. They related Elo to raw and evolution-weighted sequence counts, then simulated entropy-regularised MCMC design to measure drift and effects on predicted Tm and pI.
- Swiss-Prot orthologous protein sets from 133 species (203 protein types, 40% eukaryotic) · multiple species · n = 7545 sequences
- Simulated protein designs from pLM-likelihood MCMC across many species' orthologs as starting sequences · multiple species · n = 1092 designs
- Methods & tools
- Progen2 (xlarge, BFD90, large, base, medium), ESM2 (15B, 3B, 650M), Elo rating system, TimeTree of Life, UniProt / Swiss-Prot / UniRef, Protein BLAST, DeepSTABp, Bio.Align (Smith-Waterman, BLOSUM62), Gibbs sampling MCMC
- Reproducibility gaps
- The text gives no code repository or dataset link. Source data are public databases (Swiss-Prot via the UniProt REST API), but no URL is printed.
Limitations (5)
- Thermostability and salt tolerance were assessed only in silico (DeepSTABp Tm and calculated pI), so the design conclusions depend on predictor accuracy; no experimental assays were run. · stated by the authors
- The evolution-weighted count assumes a fixed mutation rate, but mutation rates vary across species and proteins. · stated by the authors
- Only likelihoods were studied; whether embeddings or other protein model types carry the same bias, or keep it after fine-tuning, is left open. · stated by the authors
- The design experiments use one model (Progen2-xlarge) and small, hand-selected protein sets (13 for thermostability; 18 ribosomal proteins from 3 halophilic species for salt), so generality of the design effects is untested. · noted in this summary
- No bias-mitigation method is implemented or tested; the correction ideas are only proposed in the conclusion. · noted in this summary
Who should read:Read this if you use protein language model likelihoods to score or design sequences, especially from under-sampled lineages such as extremophiles.
Reproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- Protein Language Model Fitness Is a Matter of Preference
Cade Gordon, Amy X. Lu, Pieter Abbeel·bioRxiv·Oct 3, 2024
- Protein Language Models: Is Scaling Necessary?
Quentin Fournier, Robert McCoy Vernon, … CHRISTOPHER JAMES LANGMEAD·bioRxiv·Sep 23, 2024
- Pretrained protein language models choose between sequence novelty and structural completeness
Arjuna M. Subramanian, Zachary A. Martinez, Matt W. Thomson·bioRxiv·Oct 1, 2025
- Language modelling for biological sequences - curated datasets and baselines
José Juan Almagro Armenteros, Alexander Rosenberg Johansen, … Henrik Nielsen·bioRxiv·Mar 9, 2020
- Protein Language Models: Learning From Evolution, Designing Beyond It
Maurice Brenner, Julius Schlensok, … Michael Heinzinger·bioRxiv·Sep 24, 2026
- Simple baselines rival protein language models in dense combinatorial protein landscapes
Itay Talpir, Sarel Jacob Fleishman·bioRxiv·May 8, 2026