Simulating 500 million years of evolution with a language model

Key results

1. ShownChain-of-thought prompting of base ESM3 7B and two 96-well plate screens generated esmGFP, a bright green fluorescent protein 96 mutations away from its closest known neighbor.

   Conditioned on 16 chromophore/helix residues from PDB 1QY3, round 1 (88 E. coli designs) yielded dim hit B8 (57% identity to tagRFP, 50x dimmer than natural GFPs) and round 2 (90 designs from B8) yielded esmGFP (well C10), sharing 58% identity (96 mutations, 22 internal) with tagRFP, 53% (107 mutations) with eqFP578, and 36% with avGFP while matching natural GFP brightness at 2 days (496/512 nm peaks). · Abstract; Generating a new fluorescent protein; A.5 GFP; Figure legends

2. ShownBiological alignment via Iterative Reasoning Preference Optimization (IRPO) doubled tertiary motif scaffolding success rates, with the largest gains emerging at the 98B parameter scale.

   Across 46 held-out BioLiP ligand-binding motifs (1,024 permuted prompts per motif), IRPO finetuning on 30,000 prompt preference pairs increased Pass@128 success (backbone cRMSD < 1.5 A and ESMFold pTM > 0.8) from 9.5% to 18.8% (1.4B), 19.0% to 37.4% (7B), and 26.8% to 65.5% (98B), surpassing supervised finetuning (14.2%, 33.7%, and 44.6%). · Biological alignment; A.4.1. Algorithm; A.4.5. Evaluation Dataset: Tertiary Coordination; A.4.6 Supervised Finetuning

3. ShownDiscrete VQ-VAE structure tokenization achieved sub-angstrom reconstruction, and scaling ESM3 to 98B parameters surpassed ESMFold in single-sequence structure prediction.

   The 4,096-code VQ-VAE tokenizer reconstructed CAMEO test structures to <0.5 A all-atom RMSD (backbone RMSD < 1 A, LDDT-CA > 0.98), and ESM3 98B achieved 0.880 mean LDDT vs 0.861 for ESMFold on CAMEO while generating diverse unconditional proteins (mean pLDDT 0.84, pTM 0.52, mean pairwise sequence identity 0.155, mean TM-score 0.48). · ESM3; A.1.7 Structure Tokenizer; A.1.7.4 Evaluation; Tables

4. ShownMultimodal prompt composition generated designable scaffolds distinct from natural folds and compressed a serine protease by one-third while preserving active-site geometry.

   Out-of-distribution SS8/SASA prompts yielded globular proteins (mean pTM 0.85 +/- 0.03) with <20% sequence identity and 0.48 +/- 0.09 TM-score to training data; composing atomic motifs with fold/SS8 prompts gave median pTM 0.80 +/- 0.08 and scTM 0.96 +/- 0.04, and compressing trypsin (PDB 1Y3V) from 223 to 150 residues kept the catalytic triad at 0.73 A all-atom RMSD (pTM 0.84, scTM 0.97 +/- 0.006). · Programmable design with ESM3; A.3.9. Steerable Design; A.3.10. Composing Prompts; A.3.11. Multimodal Editing Examples

5. ShownData filtering in the open-weights 1.4B ESM3-open model ablated zero-shot viral fitness prediction to baseline levels without degrading non-viral representation or structure performance.

   Removing 10.6 million viral and Select Agent sequences and 9,462 of 68,103 function keywords reduced Spearman rho across 28 ProteinGym viral deep mutational scanning assays from 0.28 (ESM3-small) to 0.17 (ESM3-open, matching BLOSUM62), while non-viral DMS Spearman rho remained 0.45 vs 0.46 and function keyword macro mAP reached 0.81. · A.6 OPEN MODEL; Filtering sequences of potential concern; Function Keyword Prediction; Zero-shot Viral Fitness Prediction