henhaobio
Sign in

bioRxivMicrobiologyPreprint

A contextualised protein language model reveals the functional syntax of bacterial evolution

Maciej Wiatrak, David C. Abelson, Roman Mikhaylenko, Ramon Viñas Torné, Maria Ntemourtsidou, Adam M. DinaniD, Divya Arora, Samuel HorsfieldiD, John A. LeesiD, Maria BrbićiD, Aaron WeimanniD, R. Andres FlotoiD

PDFDOICodeData
In-depth full-text digest available · Jump to digest ↓

Abstract

Bacteria have evolved a vast diversity of functions and behaviours that are currently incompletely understood and poorly predicted from DNA sequence alone. To understand the syntax of bacterial evolution and discover genome-to-phenotype relationships, we curated over 1.3 million genomes spanning bacterial phylogenetic space, represented each as an ordered sequence of proteins, and used these sequences to train a transformer-based, contextualised protein language model, Bacformer. By pretraining on genome-wide evolutionary patterns, Bacformer captures the compositional and positional relationships of proteins and thereby provides a whole-genome framework for linking genomic organisation and content to measurable bacterial traits. We demonstrate the ability of Bacformer to accurately predict protein-protein interactions; uncover operon structure, which we validated experimentally; infer important phenotypic traits, including antimicrobial resistance, while revealing likely causal genes; and design template synthetic proteomes with desirable properties. Thus, Bacformer establishes a genomic foundation model that reveals the evolutionary rules governing bacterial gene organisation, function, and phenotype, opening a route to systematic whole-genome engineering.

In-depth summary

Claude Opus 5.5 · xhigh effort · 2 Nov 2026 · from the full text of version 3 (19 Aug 2026) · CC BY 4.0

Trained on over 1.3 million bacterial genomes represented as ordered protein sequences, the transformer foundation model Bacformer predicts protein interactions, operons, essentiality, and phenotypes while generating trait-conditioned synthetic proteomes.

Existing DNA and protein language models incorporate limited genomic context and require splitting whole bacterial genomes into segments, making full-genome finetuning memory-prohibitive and ignoring genome-wide functional organization. By representing each protein as a single token ordered along contigs and chromosomes across over 1.3 million metagenome-assembled and complete genomes, Bacformer captures whole-proteome synteny and co-evolution in a single pass with single-digit-GB GPU memory.

Key results

  1. ShownBacformer embeddings accurately cluster bacterial genomes across taxonomic ranks and resolve strain-level subpopulations in an unsupervised manner, outperforming non-contextual protein language models and sequence identity metrics.

    On 6,071 MAGs across 25 species, Bacformer-L achieved mean NMI of over 0.98 +/- 0.01 and ARI of 0.97 +/- 0.02 versus 0.86 +/- 0.01 and 0.71 +/- 0.02 for ESM-C; kNN retrieval across 1,314,344 MAGs reached 0.83 +/- 0.32 species, 0.94 +/- 0.17 genus, and 0.95 +/- 0.16 family balanced accuracy. · Genome embeddings uncover evolutionary…; Benchmarking; Scalable search for similar bacteria; Figure legends

  2. ShownBacformer captures genome-wide protein-protein interactions (PPIs) and operon boundaries zero-shot and upon finetuning, relying strongly on conserved gene order.

    Bacformer-L improved zero-shot STRING PPI AUROC by 14 percentage points over ESM-C on held-out genera (shuffling gene order dropped AUROC by 13 percentage points, p < 10^-5), prioritized P. aeruginosa pairs with higher AlphaFold3 ipTM scores than distance-matched controls (p < 0.004), and predicted operons across 5 species with 0.76-0.92 zero-shot AUROC. · Bacformer reveals functional organisation of…; Benchmarking; Identifying protein–protein interactions in…; Zero-shot operon prediction using long-read RNA…

  3. ShownContextualized protein embeddings accurately predict gene essentiality across unseen genera and KEGG Orthology protein functions, enabling functional annotation of previously unannotated proteins.

    Across 193,213 genes in 55 DEG genomes (genus-level split), a linear probe on Bacformer embeddings achieved 0.81 +/- 0.07 median AUROC (0.64 when masking the query protein itself); across 300 genomes (9,059 KEGG KO groups), balanced accuracy exceeded 94% (linear probe) and 98.5% after full-model finetuning, annotating 98,821 unannotated proteins across 60 test strains. · Predicting protein essentiality and functions; Gene essentiality prediction and benchmarking; Protein function prediction; Annotating proteins with unknown function

  4. ShownWhole-genome embeddings predict 138 bacterial phenotypic traits and antimicrobial resistance (AMR) across dozens of drugs while attributing predictions to known and candidate causal genes.

    On 26,052 NCBI AST genomes (37 binary and 56 MIC regression antibiotics), Bacformer-L improved AUPRC by over 12 percentage points compared to ESM-C (randomizing gene order reduced AUPRC by over 12 percentage points, p < 10^-10); gradient and ablation attribution recovered canonical sporulation (spoIIIAG, spoIIAF), niche-specific, and AMR genes (p < 10^-10). · Genome-scale inference of bacterial traits and…; Predicting antimicrobial resistance (AMR) and…; Identifying phenotype-specific genes; Identifying AMR-associated genes

  5. ShownAutoregressive and contrastive variants of Bacformer recover intact genome embeddings from incomplete proteomes and generate diverse synthetic proteome templates conditioned on phenotypic traits.

    On 2,086 unseen complete genomes, autoregressive Bacformer achieved median perplexity of 9.12 (vs 50,000 random), extrapolated full genomic context when 50% of proteins were ablated (N = 9,688), and enriched aerobic (pucL, CYC) or psychrophilic (xynB, bgaB) KEGG groups when generating 20 genomes from 500-protein prompts conditioned on trait tokens. · Generating de novo bacterial proteomes; Generating genomes with a broad range of protein…; Genome completion; Generating genomes with desired properties with…

Methods, data and reproducibility

Genomes are modeled as ordered protein sequences (up to 6,000 tokens) using mean-pooled ESM-2 or ESM-C embeddings with contig tokens and rotary positional embeddings in 26M-parameter (12-layer) or 332M-parameter (30-layer) transformers. Models were pretrained via masked or autoregressive prediction of 50,000 k-means protein clusters on 1,314,344 MAGs and 48,932 complete genomes, then benchmarked on BacBench.

  • Quality-filtered metagenome-assembled genomes (MAGs) from MGnify v2.0 (306,596 genomes) and SPIRE (1,008,689 MAGs) across 70 environments used for pretraining · bacteria (other) · n = 1,314,344 genomes (2,700,709,306 proteins, 24,663 species)
  • Unique complete bacterial isolate genomes from NCBI RefSeq and GenBank used for second-stage pretraining · bacteria (other) · n = 48,932 complete genomes
  • Long-read Oxford Nanopore cDNA-PCR RNA-seq of 5 species (S. aureus RN450, M. abscessus ATCC 19977, M. tuberculosis H37Rv 102J23, P. aeruginosa PAO1, E. coli DH5alpha) for operon ground truth · E. coli, bacteria (other) · n = 5 bacterial strains (3 biological replicates each) · GCF_000013425.1, GCF_000195955.2, GCF_000069185.1, GCF_000006765.1, GCF_002899475.1
  • Bacterial protein-protein interaction datasets from STRING DB v12.0 (full evaluation and 261-strain genus-split benchmark) · bacteria (other) · n = 10,533 strains across 6,956 species (2,088 held-out test strains)
  • Gene essentiality dataset from Database of Essential Genes (DEG v15) mapped to NCBI GenBank assemblies · bacteria (other) · n = 55 genomes across 37 species (193,213 genes: 21,626 essential)
  • Protein function annotation dataset of bacterial strains from KEGG and NCBI RefSeq · bacteria (other) · n = 300 genomes (1,126,510 proteins; 590,657 annotated across 9,059 KO groups)
  • Curated categorical bacterial phenotypic traits dataset matched to NCBI GenBank assemblies and applied to human gut metagenome sample SRS1996016 · bacteria (other) · n = 24,462 genomes from 15,477 species across 138 traits; 5,811 gut MAGs · SRS1996016
  • Antimicrobial susceptibility test (AST) isolate genomes from NCBI AST Browser across 37 pathogenic species · E. coli, bacteria (other) · n = 26,052 genomes (37 binary S/R antibiotics; 56 MIC regression antibiotics)
Methods & tools
Bacformer, BacBench, ESM-2, ESM-C, gLM2, Evo, Evo-2, AlphaFold3, MMseqs2, Panaroo, AMR Finder Plus, Scanpy
Software & environment
PyTorch with FlashAttention and bfloat16 mixed precision, HuggingFace Transformers, CuPy, NVIDIA cuML, Scanpy, Captum v0.8, Apache 2.0 license
Assay conditions & compute
ESM-2 embedding of 1.3M genomes: ~30 days on 16 Intel Ponte Vecchio GPUs (~5.6 TB); pretraining: ~2 weeks on 4 NVIDIA A100 GPUs (or H100s for Bacformer-L); BacBench tasks timed on 1 NVIDIA H100 (96 GB) with 72 Grace CPU cores
Limitations (5)
  • Publicly available complete-genome and isolate databases are heavily biased toward clinically important human pathogens, leaving broad sections of the global bacterial ecosystem underrepresented. · stated by the authors
  • Novel physical protein-protein interactions prioritized by Bacformer-L in Pseudomonas aeruginosa were screened computationally with AlphaFold3 rather than experimentally validated. · stated by the authors
  • De novo generated proteome templates and trait-conditioned protein family sequences were evaluated in silico by KEGG functional enrichment; experimental synthesis and validation of synthetic genome viability remain outside the study's scope. · stated by the authors
  • Contrastive retrieval of exact underlying protein sequences from contextualized embeddings performs substantially worse at low K values when the query protein is masked and must be inferred solely from genomic context. · stated by the authors
  • Unlike the genus-held-out splits used for PPI, essentiality, and phenotype benchmarks, antimicrobial resistance models were evaluated using random 70/10/20% train/validation/test splits without phylogenetic stratification. · noted in this summary

Read this if you build genomic or protein foundation models, predict bacterial traits, operons, or antimicrobial resistance from whole-genome context, or design synthetic microbial proteomes.

Citations
15

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISSethi '25Sethi '26Toibazar '24Urhan '23Mordret '25Shao '24
72% semantic matchGenetics
Palash Sethi, Marc G. Chevrette, Juannan Zhou·bioRxiv·Mar 19, 2025· 1 citations