bioRxivBioinformaticsPreprint
Structure-informed Language Models Are Protein Designers
Zaixiang ZhengiD, Yifan DengiD, Dongyu XueiD, Yi ZhouiD, Fei YeiD, Quanquan GuiD
Abstract
This paper demonstrates that language models are strong structure-based protein designers. We present LM-DO_SCPLOWESIGNC_SCPLOW, a generic approach to reprogramming sequence-based protein language models (pLMs), that have learned massive sequential evolutionary knowledge from the universe of natural protein sequences, to acquire an immediate capability to design preferable protein sequences for given folds. We conduct a structural surgery on pLMs, where a lightweight structural adapter is implanted into pLMs and endows it with structural awareness. During inference, iterative refinement is performed to effectively optimize the generated protein sequences. Experiments show that LM-DO_SCPLOWESIGNC_SCPLOW improves the state-of-the-art results by a large margin, leading to 4% to 12% accuracy gains in sequence recovery (e.g., 55.65%/56.63% on CATH 4.2/4.3 single-chain benchmarks, and >60% when designing protein complexes). We provide extensive and in-depth analyses, which verify that LM-DO_SCPLOWESIGNC_SCPLOW can (1) indeed leverage both structural and sequential knowledge to accurately handle structurally non-deterministic regions, (2) benefit from scaling data and model size, and (3) generalize to other proteins (e.g., antibodies and de novo proteins).
Abstract summary
Claude Opus 5.5 · xhigh effort · 18 Oct 2026 · from the abstract · CC BY-NC-ND 4.0
Implanting a lightweight structural adapter into a pretrained protein language model, with iterative refinement at inference, raises sequence recovery for given folds to 55.65%/56.63% on CATH 4.2/4.3.
Structure-based protein design asks for a sequence that folds to a given structure, while the evolutionary knowledge accumulated by sequence-based protein language models (pLMs) sits in models that first have to be reprogrammed for that job. The argument here is that such a model can be turned into a structure-conditioned designer by grafting on a small adapter instead of training a new architecture, and that doing so improves on the previous best sequence recovery by 4% to 12%.
Key results
ShownThe adapted language model recovers native sequence better than the previous best structure-based designers on standard fold-conditioned benchmarks, and the margin holds up on multi-chain targets.
Sequence recovery rises by 4% to 12% over the prior best: 55.65% on the CATH 4.2 single-chain benchmark, 56.63% on CATH 4.3, and above 60% for complexes. · Abstract
SuggestedStructure awareness is obtained by surgery on an existing pLM - a lightweight structural adapter inserted into the pretrained sequence model - with sequences then polished by iterative refinement during inference rather than generated in one pass.
Presented as the design of the method; the abstract reports no ablation separating the adapter's contribution from that of iterative refinement, and no parameter counts or training cost. · Abstract
SuggestedFollow-up analyses are reported to confirm that the model uses sequence as well as structural information in regions the backbone does not determine, that performance rises with more data and larger models, and that it transfers to antibodies and de novo proteins.
Listed as three verified analyses; no figures, scaling curves or per-category recovery values accompany them in the abstract. · Abstract
Methods, data and reproducibility
A pretrained sequence-based protein language model is fitted with a lightweight structural adapter so it can condition on a target fold, and the sequences it generates are improved by iterative refinement at inference time. Performance is measured as sequence recovery on CATH 4.2 and CATH 4.3 single-chain benchmarks and on protein complexes, with further analyses on antibodies and de novo proteins.
- CATH single-chain design benchmarks, versions 4.2 and 4.3
- Protein complexes used to test design beyond single chains
- Antibodies and de novo proteins used to probe generalization
- Methods & tools
- LM-Design, CATH
Limitations (2)
- The baselines being improved upon are never named, and the 4% to 12% range is not tied to particular competing methods, so the comparison cannot be checked from the text available. · noted in this summary
- Quality is judged by sequence recovery against the native sequence; no folding, stability or experimental validation of designed sequences is reported here, and the generalization and scaling claims come without numbers. · noted in this summary
Read this if you design protein sequences for a target fold and want to know whether a repurposed sequence language model outperforms the previous best structure-based designers on sequence recovery.
- Citations
- 66
Reproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- Adapting protein language models for structure-conditioned design
Jeffrey A. Ruffolo, Aadyot Bhatnagar, … Ali Madani·bioRxiv·Aug 3, 2024
- Language models generalize beyond natural proteins
Robert Verkuil, Ori Kabeli, … Alexander W. Rives·bioRxiv·Dec 21, 2022
- InstructPLM: Aligning Protein Language Models to Follow Protein Structure Instructions
Jiezhong Qiu, Junde Xu, … Guangyong Chen·bioRxiv·Apr 17, 2024
- Bilingual Language Model for Protein Sequence and Structure
Michael Heinzinger, Konstantin Weißenow, … Burkhard Rost·NAR Genomics and BioinformaticsPeer-reviewed·Sep 28, 2024
- Structure-conditioned masked language models for protein sequence design generalize beyond the native sequence space
D. Akpinaroglu, K. Seki, … T. Kortemme·bioRxiv·Dec 15, 2023
- SaProt: Protein Language Modeling with Structure-aware Vocabulary
Jin Su, Chenchen Han, … Fajie Yuan·bioRxiv·Oct 1, 2023