henhaobio
Sign in

Nature MethodsGenomicsPeer-reviewed journal article

Effective gene expression prediction from sequence by integrating long-range interactions

Žiga AvseciD, Vikram AgarwaliD, Daniel Visentin, Joseph R. E. LedsamiD, Agnieszka Grabska‐Barwińska, Kyle R. TayloriD, Yannis AssaeliD, John JumperiD, Pushmeet KohliiD, David R. KelleyiD

PDFDOIPreprintCodeData

In-depth summary

Claude Opus 5.5 · max effort · 8 Nov 2026 · from the full text (7 Apr 2021) · CC BY 4.0

Enformer, a sequence-based transformer integrating elements up to 100 kb away, lifts held-out gene expression prediction (correlation 0.81 to 0.85 versus Basenji2) and variant effect prediction.

Convolutional sequence-to-expression models such as Basenji2 and ExPecto only reach elements within 20 kb of the transcription start site (TSS), yet many enhancers, repressors and insulators act from farther away, capping accuracy for interpreting noncoding disease variants. Enformer swaps dilated convolutions for self-attention to reach 100 kb, raising the share of high-confidence enhancer-gene pairs in view from 47% to 84%.

Key results

  1. ShownEnformer predicts Cap Analysis Gene Expression (CAGE) signal at transcription start sites of held-out human protein-coding genes more accurately than Basenji2, with gains for histone, transcription factor binding and accessibility tracks, and outperforms ExPecto on RNA-seq expression.

    Mean test-set Pearson r across protein-coding genes 0.85 vs 0.81 (Basenji2; bootstrap s.d. 0.004), a third of the gap to the replicate-based ceiling of 0.94; gains in all 4 assay types over 5,313 tracks (paired Wilcoxon P < 10^−38). Vs ExPecto (human-only Enformer plus linear model): Spearman r 0.850 vs 0.812 across genes and 0.451 vs 0.368 across tissues. · Enformer improves gene expression prediction; Model training and evaluation; Figure legends

  2. ShownEnformer variant predictions track GTEx expression quantitative trait locus (eQTL) statistics better than Basenji2 after accounting for linkage disequilibrium, chiefly in tissue-matched CAGE samples, and better separate fine-mapped causal eQTLs from matched non-causal variants.

    Signed LD profile (SLDP) vs Basenji2: max Z-score up for 379 CAGE datasets (59.4%), by more than one s.d. for 228 vs 46 down; mean 6.3 to 6.9; DNase Z above DeepSEA Beluga for 60.8% of pairs. Random forests (SuSiE causal probability > 0.9 vs < 0.01): better in 47 of 48 GTEx v8 tissues, mean auROC 0.729 to 0.747, at all TSS distances (one-sided paired Wilcoxon P < 1 × 10^–4). · Enformer improves variant effect prediction on…; GTEx SLDP; Fine-mapped GTEx classification; Figure legends

  3. ShownOn CAGI5 saturation-mutagenesis massively parallel reporter assay (MPRA) data, per-locus lasso models on Enformer allele-difference features achieved the best average correlation of any compared entry, and training-free Enformer CAGE/DNase scores performed comparably.

    Across 15 loci, lasso-Enformer ranked first against seven alternative submissions and beat CAGI5 winner Group 3 (P = 0.002, paired one-sided Mann-Whitney U). Training-free scores also beat deltaSVM, trained on matched-cell DNase and H3K4me1. At LDLR, Enformer recovered two of four binding sites; deltaSVM caught one, with 50% lower Pearson and Spearman correlations. · Enformer improves MPRA mutation effect…; Benchmarking variant effect predictions on…; Figure legends

  4. ShownFrom sequence alone, Enformer contribution scores rank CRISPR interference (CRISPRi)-validated enhancer-gene pairs in K562 above Basenji2 and random, rivaling the ABC score that requires Hi-C and H3K27ac; attention rises at and drops across topologically associating domain (TAD) boundaries.

    auPRC on Gasperini (scRNA-seq readout) and Fulco (Flow-FISH) K562 screens covering >10,000 perturbed candidate enhancers: Enformer beats Basenji2 and random at almost all distance bins; Basenji2 scores are zero beyond 20 kb; cell-type-specific scores beat agnostic ones. TAD attention patterns significant over 1,500 sequences (two-sided Mann-Whitney U). · Enformer attends to cell-type-specific enhancers; Enhancer prioritization; Figure legends

  5. ShownAblations tie the gain to attention: transformer layers beat dilated convolutions at all tested model sizes, depths and training-set sizes; restricting Enformer to Basenji2's receptive field cut accuracy sharply; custom relative positional encodings beat sin/cos and absolute ones.

    Comparisons mostly used validation-set CAGE gene expression correlation and matched 768-channel models on 131 kb inputs without human fine-tuning; a local-attention variant with Basenji2's 44 kb receptive field (fixed radius 16) dropped markedly; symmetric-only positional bases underperformed. Effect sizes are given only in Extended Data figures. · Enformer improves gene expression prediction; Model training and evaluation; Figure legends

Methods, data and reproducibility

Enformer stacks 7 convolution/attention-pooling blocks and 11 transformer blocks with custom relative positional encodings over 196,608 bp of one-hot DNA, jointly predicting 5,313 human and 1,643 mouse tracks in 128-bp bins (Poisson loss); allele prediction differences and contribution scores are benchmarked on CRISPRi, eQTL and MPRA data.

  • Human Basenji2-derived set on hg38 (196,608 bp inputs): 2,131 TF ChIP-seq, 1,860 histone ChIP-seq, 684 DNase/ATAC-seq and 638 CAGE tracks · human · n = 34,021 train, 2,213 validation, 1,937 test sequences
  • Mouse Basenji2-derived set trained jointly with human: 308 TF ChIP-seq, 750 histone ChIP-seq, 228 DNase/ATAC-seq and 357 CAGE tracks · mouse · n = 29,295 train, 2,209 validation, 2,017 test sequences
  • K562 CRISPRi enhancer-gene tests (Gasperini et al., scRNA-seq; Fulco et al., Flow-FISH) plus ENCODE K562 H3K27ac and DNase data for the ABC baseline · human (K562 cell line) · n = >10,000 candidate enhancers perturbed · GSE120861 (Gasperini); ENCFF779QTH, ENCFF413AHU, ENCFF936BDN (ENCODE)
  • TAD boundaries processed by Fudenberg et al. 2020, used to center sequences for attention-matrix and TF-MoDISco motif analyses · n = 1,500 boundary-centered vs 1,500 control sequences
  • GTEx v7a eQTL summary statistics tested by SLDP against Enformer scores for all 1000 Genomes SNPs · human (GTEx, 1000 Genomes) · n = 48 tissues; 638 CAGE samples
  • GTEx v8 eQTLs fine-mapped by SuSiE (Wang et al. 2021) with matched likely non-causal negatives for random forest classification · human (GTEx) · n = 48 tissues; 166 to 2,740 causal variants per tissue
  • CAGI5 saturation mutagenesis MPRA of enhancers and promoters, with competition training/test splits and participant accuracies · human (assayed in cell lines such as HepG2, K562, HEK293T) · n = 15 loci
  • RNA-seq expression resources distributed with ExPecto, used for across-gene and across-tissue comparison against a human-only retrained Enformer · human · n = 990 test genes
Methods & tools
Enformer (Sonnet v2/TensorFlow; code archived at Zenodo 10.5281/zenodo.5098375), Basenji2 (pretrained baseline; retrained 768-channel variant for ablations), ExPecto and DeepSEA Beluga (baselines), ABC score (Fulco et al.; Hi-C/H3K27ac/DNase-based enhancer baseline), SLDP (signed linkage disequilibrium profile) regression, SuSiE fine-mapping, scikit-learn 0.22 random forest, glmnet cv.glmnet lasso (R), TF-MoDISco with HOCOMOCO v11 and Tomtom, deltaSVM, CADD and other CAGI5 entries (comparators), UCSC liftOver and hg38-mm10 syntenic net alignment, Gradient × input, attention and in silico mutagenesis contribution scores
Software & environment
Sonnet v2 on TensorFlow v2.4.0 (Adam, learning rate 0.0005, 5,000-step warm-up); scikit-learn 0.22 random forests; glmnet cv.glmnet in R; hg38 reference with hg38-mm10 syntenic splits; GENCODE v32; pretrained model on TF-Hub; Apache 2.0 code.
Assay conditions & compute
Main 1,536-channel model: 64 TPU v3 cores, batch size 64, 150,000 steps (approximately 3 days) plus 30,000 human fine-tuning steps; ablation models: 32 TPU v3 cores, batch size 32; ablation depth and width limited by TPU memory.
Reproducibility gaps
CAGI5 MPRA sets were obtained via personal communication; Kipoi release only planned; released 1000 Genomes scores are PCA-compressed to 20 scores (fine-mapping auROC 0.743 vs 0.747 with all features); SLDP details are deferred to Supplementary Information.
Limitations (6)
  • Predictions are restricted to the cell types and assays in the training compendium; the model cannot generalize to unseen cell types or assays, which the authors suggest representation learning of cell types and assays could address. · stated by the authors
  • Sign prediction for fine-mapped eQTLs remains weak for both Enformer and Basenji2 once variants lie beyond the promoter (TSS distance > 1,000), which the authors flag as an open problem. · stated by the authors
  • The 128-bp output resolution is a compromise: finer bins would lengthen the sequence fed to quadratic-cost self-attention beyond what current hardware supports, and ablation depth and width were capped by TPU memory. · stated by the authors
  • In the treated-sample analysis, the authors note that fold changes between CAGE samples that are already highly correlated, such as differently perturbed samples within one group, are harder for the model to predict. · stated by the authors
  • The headline 0.81 to 0.85 gain pits the full model against pretrained Basenji2 while changing several factors at once (attention, attention pooling, doubled channels, 1.5 times longer input, human fine-tuning); the isolating ablations used smaller, non-fine-tuned models. · noted in this summary
  • Some benchmarks are narrow or retrospective: enhancer prioritization is tested only in K562, and the CAGI5 comparison is a post hoc evaluation rather than a blind entry, with training-free scores relying on manually chosen cell-type-matched features per locus. · noted in this summary

Read this if you build or use sequence-to-expression models for noncoding variant interpretation or enhancer-gene linking and want to see what a 100 kb self-attention receptive field adds over Basenji2.

Citations
1,348
FWCI
48.72
Field percentile
top 1%

Abstract

The next phase of genome biology research requires understanding how DNA sequence encodes phenotypes, from the molecular to organismal levels. How noncoding DNA determines gene expression in different cell types is a major unsolved problem, and critical downstream applications in human genetics depend on improved solutions. Here, we report substantially improved gene expression prediction accuracy from DNA sequence through the use of a new deep learning architecture called Enformer that is able to integrate long-range interactions (up to 100 kb away) in the genome. This improvement yielded more accurate variant effect predictions on gene expression for both natural genetic variants and saturation mutagenesis measured by massively parallel reporter assays. Notably, Enformer outperformed the best team on the critical assessment of genome interpretation (CAGI5) challenge for noncoding variant interpretation with no additional training. Furthermore, Enformer learned to predict promoter-enhancer interactions directly from DNA sequence competitively with methods that take direct experimental data as input. We expect that these advances will enable more effective fine-mapping of growing human disease associations to cell-type-specific gene regulatory mechanisms and provide a framework to interpret cis-regulatory evolution. To foster these downstream applications, we have made the pre-trained Enformer model openly available, and provide pre-computed effect predictions for all common variants in the 1000 Genomes dataset. One-sentence summaryImproved noncoding variant effect prediction and candidate enhancer prioritization from a more accurate sequence to expression model driven by extended long-range interaction modelling.

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISKarollus '22Lin '24Zeng '19Kribelbauer '24Ghosal '25Drusinsky '26
62% semantic matchBioinformatics
Alexander Karollus, Thomas Mauermeier, Julien Gagneur·bioRxiv·Sep 15, 2022· 14 citations