henhao
Sign in
Synthetic BiologyScience
Peer-reviewed journal article

Sequence modeling and design from molecular to genome scale with Evo

Eric NguyeniD, Michael PoliiD, Matthew G. DurrantiD, Brian Kang, Dhruva KatrekariD, David B. LiiD, Liam J. BartieiD, Armin W. ThomasiD, Samuel H. KingiD, Garyk BrixiiD, Jeremy Sullivan, Madelena Y. NgiD, Ashley Adanna LewisiD, Aaron Lou, Stefano ErmoniD, Stephen A. BaccusiD, Tina Hernandez‐BoussardiD, Christopher RéiD, Patrick D. HsuiD, Brian HieiD

PDFDOIPreprintCode
Post to X
Threads on XPost to BlueskyThreads on Bluesky
In-depth full-text digest available · Jump to digest ↓

Abstract

The genome is a sequence that completely encodes the DNA, RNA, and proteins that orchestrate the function of a whole organism. Advances in machine learning combined with massive datasets of whole genomes could enable a biological foundation model that accelerates the mechanistic understanding and generative design of complex molecular interactions. We report Evo, a genomic foundation model that enables prediction and generation tasks from the molecular to genome scale. Using an architecture based on advances in deep signal processing, we scale Evo to 7 billion parameters with a context length of 131 kilobases (kb) at single-nucleotide, byte resolution. Trained on 2.7M prokaryotic and phage genomes, Evo can generalize across the three fundamental modalities of the central dogma of molecular biology to perform zero-shot function prediction that is competitive with, or outperforms, leading domain-specific language models. Evo also excels at multi-element generation tasks, which we demonstrate by generating synthetic CRISPR-Cas molecular complexes and entire transposable systems for the first time. Using information learned over whole genomes, Evo can also predict gene essentiality at nucleotide resolution and can generate coding-rich sequences up to 650 kb in length, orders of magnitude longer than previous methods. Advances in multi-modal and multiscale learning with Evo provides a promising path toward improving our understanding and control of biology across multiple levels of complexity.

02 /Specification & Classification

Click any tag to browse matching papers
Citations
593
FWCI
80.56
Field percentile
top 1%
OpenAlex Topics

In-depth summary

✦ Multi-agent verified full-text synthesis

Claude Sonnet 4.6 · max effort · 8 Nov 2026 · from the full text of version 2 (6 Mar 2024) · CC BY 4.0

7B-parameter convolution-attention DNA model with 131 kb context, trained on prokaryotic genomes, performs zero-shot fitness and essentiality prediction and generates Cas loci, transposons, and 650 kb sequences.

Dense Transformer DNA models scale quadratically with sequence length and do poorly at single-nucleotide resolution, so they rely on token aggregation and short contexts. Earlier generative biology covered only single molecules, simple complexes, or short DNA; Evo uses a hybrid attention-convolution design to unify prediction and design from molecule to genome scale.

Key results

  1. ShownIn a compute-optimal scaling study on DNA, Hyena and the attention-hybrid StripedHyena scaled better than Transformer++ and Mamba, and held up better outside the compute-optimal frontier.

    Over 300 models across four architectures (6 million to 1 billion parameters; budgets 8 × 10^18 to 8 × 10^19 FLOPS). Transformer++ had worse perplexity at all budgets; Hyena and StripedHyena had the best scaling rate; Transformer++ and Mamba showed training instability. · 2.3. StripedHyena demonstrates favorable scaling…; Hyperparameter tuning and direct model…; B.4. Scaling laws; Scaling laws procedure

  2. ShownZero-shot Evo likelihoods predict fitness effects for E. coli proteins, ncRNAs, and promoter-RBS expression, beating other nucleotide models and approaching protein language models. Prediction fails on human proteins.

    E. coli DMS: Evo beat all nucleotide LMs and was competitive with protein LMs. 5S rRNA Spearman r = 0.64 (P = 7.3×10^−4). Promoter-RBS mRNA Spearman r = 0.41 (P < 1×10^−5); binarized protein AUROC = 0.68. Evo could not predict human protein DMS. · 2.4.1. Predicting mutational effects on protein…; 2.4.2. Predicting mutational effects on ncRNA…; 2.4.3. Predicting gene expression from…; B.6.1. Protein function prediction

  3. ShownChanges in Evo log-likelihood after in silico premature stop-codon insertion predict gene essentiality across bacterial and phage genomes without supervision, and flanking genomic context improves the prediction.

    With 66k context, significant (Bonferroni-corrected permutation P < 0.05) in 43 out of 58 genomes (56 bacterial, 2 phage). AUROC 0.86 for lambda phage and 0.81 for Pseudomonas aeruginosa. GenSLM could not perform the task; gene position was not predictive. · 2.7. Predicting gene essentiality with long…; B.9. Gene essentiality prediction

  4. SuggestedFinetuned Evo generated diverse CRISPR-Cas loci and IS200/IS605 transposable elements with divergent proteins and plausible predicted structures, but validation was computational only.

    Cas finetuning on 82,430 loci: 15-45% of 8 kb generations contained Cas coding sequences; some ORFs had <40% identity to the closest training match. Among generated IS200/IS605 loci with both TnpA and TnpB: 22.5% of TnpA and 90.1% of TnpB were <50% identical to training; ESMFold pLDDT > 70 for 87.6% of TnpA vs 25.5% of TnpB. · 2.5. Generative design of CRISPR-Cas molecular…; 2.6. Generative design of transposable…; B.7. CRISPR-Cas finetuning, generation, and…; B.8. IS200/IS605 finetuning, generation, and…

  5. ShownTwenty unfinetuned ~650 kb generations reproduced the high-level coding organization of bacterial genomes but lacked conserved marker genes and complete tRNA repertoires.

    CheckM coding density was nearly as high as natural genomes and above random sequences (100 crops of 131,072 bp vs 100 GTDB genomes and 1,000 random sequences). Across ~13 megabases: 18 tRNAs (vs 35 in the ~580 kb M. genitalium genome) and no rRNAs. · 2.8. Generating DNA sequences at genome scale; B.10. Genome-scale generation and evaluation

Methods, data and reproducibility

Autoregressive next-nucleotide pretraining of a 32-block StripedHyena (29 hyena layers, 3 rotary attention layers) on OpenGenome in two stages (8k, then 131k context), followed by zero-shot likelihood scoring, finetuning for Cas and IS200/IS605 generation, and in silico mutagenesis.

  • OpenGenome pretraining set: GTDB bacterial/archaeal genomes, IMG/VR v4 prokaryotic phages (eukaryotic viruses excluded), IMG/PR plasmids · bacteria, archaea, prokaryotic phages · n = over 80, 000 bacterial and archaeal genomes; 300B tokens · GTDB v214.1
  • CRISPR-Cas loci from public genomic and metagenomic sequences, used for finetuning with Cas9/Cas12/Cas13 prompt tokens · prokaryotes (genomic and metagenomic) · n = 82,430 loci
  • IS605 and IS200 elements in natural sequence context, used for finetuning · prokaryotes (genomic and metagenomic) · n = 10,720 IS605 and 219,867 IS200 elements
  • Protein deep mutational scanning studies for zero-shot fitness prediction (E. coli and human) · E. coli; human · n = 6 E. coli studies; 6 human studies
  • ncRNA mutational scanning studies (ribozymes, tRNAs, rRNA) · not stated (includes E. coli 5S rRNA growth study) · n = 7 studies
  • Kosuri et al. promoter-RBS combinations with measured mRNA and protein expression · E. coli · n = ~12k promoter-RBS pairs
  • Gene essentiality labels from the DEG database (bacteria) and Piya et al. (phages lambda and P1) · bacteria; phage · n = 56 bacterial and 2 phage genomes · RefSeq NC_001416 (lambda), NC_005856 (P1)
  • Genome-scale Evo generations from four species prompts, assessed against GTDB genomes and random sequences · bacteria (M. genitalium, S. aureus, K. pneumoniae, E. coli prompts) · n = twenty sequences of ~650 kb
Methods & tools
StripedHyena, Transformer++, Mamba, GenSLM, Nucleotide Transformer, RNA-FM, ESM-1v / ESM-2 / ProGen2 / CARP, HMMER, MMseqs2, AlphaFold2, ESMFold, CheckM
Code
https://github.com/evo-design/evo
Software & environment
Pretraining data from GTDB v214.1, IMG/VR v4, IMG/PR; analysis with scipy, Prodigal, HMMER, MMseqs2, ESMFold, CheckM, tRNAscan-SE, barrnap; scaling baselines Transformer++, Mamba, Hyena.
Assay conditions & compute
Stage 1 on 64 Nvidia H100 GPUs, stage 2 on 128 Nvidia A100 GPUs; ~340B tokens, ~2 × 10^22 FLOPS; sequences up to 650k nucleotides generated on a single 80GB GPU.
Reproducibility gaps
No download URL printed for the compiled OpenGenome set or the custom finetuning database; full training hyperparameters are deferred to supplementary Table S2.
Limitations (5)
  • Pretraining is prokaryote-only (300B tokens), so zero-shot mutational effect prediction on human proteins is limited. · stated by the authors
  • Cas generation has moderate controllability (a Cas12 token occasionally yields Cas9), and the more novel CRISPR-Cas sequences were sampled at low frequency. · stated by the authors
  • Genome-scale samples look like a 'blurry image': many low-confidence structure predictions, few conserved marker genes, incomplete tRNA repertoire (18 tRNAs) and no rRNAs. · stated by the authors
  • Generated Cas and IS200/IS605 systems are judged only by sequence identity, pHMM hits, and predicted structures; no functional activity of designed systems is reported. · noted in this summary
  • Scaling-law models span 6 million to 1 billion parameters, so the 7B Evo is an extrapolation, and the 8k to 66k essentiality gain rests on outlier genomes. · noted in this summary

Who should read:Read this if you want to assess genome-scale DNA language modeling, hybrid architectures versus Transformers, or zero-shot prediction and generative design of Cas and transposon systems.

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISBrixi '25Merchant '25Consens '26Brixi '26Hie '26Tzanakakis '26
74% semantic matchGenomics
Garyk Brixi, Matthew G. Durrant, Jerome Ku+49 more·bioRxiv·Feb 18, 2025· 218 citations