Leveraging information in spatial transcriptomics to predict super-resolution gene expression from histology images in tumors
Abstract
Recent developments in spatial transcriptomics (ST) technologies have enabled the profiling of transcriptome-wide gene expression while retaining the location information of measured genes within tissues. Moreover, the corresponding high-resolution hematoxylin and eosin-stained histology images are readily available for the ST tissue sections. Since histology images are easy to obtain, it is desirable to leverage information learned from ST to predict gene expression for tissue sections where only histology images are available. Here we present HisToGene, a deep learning model for gene expression prediction from histology images. To account for the spatial dependency of measured spots, HisToGene adopts Vision Transformer, a state-of-the-art method for image recognition. The well-trained HisToGene model can also predict super-resolution gene expression. Through evaluations on 32 HER2+ breast cancer samples with 9,612 spots and 785 genes, we show that HisToGene accurately predicts gene expression and outperforms ST-Net both in gene expression prediction and clustering tissue regions using the predicted expression. We further show that the predicted super-resolution gene expression also leads to higher clustering accuracy than observed gene expression. Gene expression predicted from HisToGene enables researchers to generate virtual transcriptomics data at scale and can help elucidate the molecular signatures of tissues.
02 /Specification & Classification
Click any tag to browse matching papers- Citations
- 130
- Organism
- Assay
- Research Focus
- Approach
In-depth summary
Claude Opus 5.5 · xhigh effort · 17 Oct 2026 · from the full text of version 1 (28 Nov 2021) · CC BY-NC-ND 4.0
Modelling all spots of a section jointly with a Vision Transformer, HisToGene predicts expression from H&E images better than a re-implemented ST-Net and adds 4x super-resolution prediction.
Spatial transcriptomics (ST) is costly while hematoxylin and eosin (H&E) slides are cheap and routine, so predicting spatial expression from histology could yield virtual ST data at scale. Unlike ST-Net, a convolutional network that treats each spot independently, HisToGene models dependency between spots through positional embeddings and attention, and the authors describe it as the first method to predict super-resolution expression from histology.
Key results
ShownHisToGene's spot-level predictions correlate with observed expression, while those of a re-implemented ST-Net show mean correlations near zero in most of the 32 breast cancer sections.
32-fold leave-one-section-out cross-validation (9,612 spots, 785 genes). Best-predicted genes by HisToGene: GNAS mean R = 0.32, MYL12B 0.27, FASN 0.27, CLDN4 0.26; ST-Net's top 4 (IGHM, PPP1R1B, IGLC2, PNMT) have much lower mean correlations. Per-section correlation values appear only in a figure that is not in the text. · Overview of evaluations; HisToGene enables super-resolution gene…; Visualization of the predicted gene expression
ShownDensely sampled image patches give 4x super-resolution expression maps, and summing sub-patches back to spot level (HisToGene*) matches observed expression better than direct spot-level prediction in most sections.
Across 32 sections, HisToGene* correlations were significantly higher than HisToGene's in 19 (59%) and significantly lower in 6 (19%). FN1 mean R = 0.24 (HisToGene*) vs 0.22 (HisToGene). Super-resolution maps of GNAS, FN1, MYL12B and FASN showed finer spatial patterns, judged visually. · Predicting gene expression at super-resolution; HisToGene enables super-resolution gene…; Visualization of the predicted gene expression
SuggestedK-means clustering of predicted expression recovers pathologist-annotated regions: aggregated super-resolution predictions gave the best adjusted Rand index (ARI) in 4 of 6 annotated sections, also beating clustering of observed expression in 4 of 6.
HisToGene* best for B1, C1, D1 and F1 (well above observed expression and ST-Net for D1 and F1); ST-Net best for E1; observed expression best for G2. ARI values are in a figure not included in the text. Clusters from the full super-resolution output (HisToGene_SR) could not be scored by ARI and were assessed visually. · HisToGene predicted gene expression can recover…
SuggestedOn a smaller cutaneous squamous cell carcinoma (cSCC) dataset mixing Spatial Transcriptomics and 10x Visium sections, neither model predicted expression reliably, though HisToGene stayed more accurate than ST-Net.
12 sections from 4 patients, 6,630 spots and 134 genes after the same filtering, leave-one-out cross-validation. The results are reported in Supplementary Note 1, which is not in the text, so no numbers are available here. · DISCUSSION
SuggestedAttention maps indicate that shallow layers focus on the target spot, middle layers begin to draw on distant spots and deep layers on distant tumour-related ones, with finer use of neighbouring patches in the super-resolution setting.
Randomly chosen maps from layers 1, 4 and 8 (8 layers x 16 heads, 128 maps) for one target spot; mean super-resolution attention weight about 1/10 of the original-resolution weight. Qualitative inspection only. · Understanding the HisToGene prediction with…
Methods, data and reproducibility
HisToGene cuts an H&E patch at each ST spot, adds learnable one-hot x/y coordinate embeddings, and feeds all spots of a section through a modified Vision Transformer (8 attention layers, 16 heads) trained on log-normalised UMI counts; averaging predictions over densely sampled overlapping patches gives 4x resolution. Compared with a re-implemented ST-Net by leave-one-section-out cross-validation.
- HER2-positive breast tumour ST data with H&E images; sections with at least 180 spots kept, leaving 9,612 spots and 785 genes after filtering · human · n = 32 sections from 7 patients (of 36 sections from 8 patients)
- Pathologist-annotated subset of the HER2+ sections, used to score clustering · human · n = 6 sections
- Cutaneous squamous cell carcinoma data from Spatial Transcriptomics and 10x Visium; 6,630 spots and 134 genes after filtering · human · n = 12 sections from 4 patients · GSE144240
- Methods & tools
- HisToGene, Vision Transformer, ST-Net, PyTorch, K-means, fgsea, MSigDB
- Reagents & environment
- Implemented in PyTorch (version not given); stated hyper-parameters: learning rate 10-5, 100 epochs, dropout 0.1, 8 multi-head attention layers, 16 heads; 112 x 112 pixel patches for the HER2+ data.
- Assay conditions & compute
- Training on 31 HER2+ sections took 11 minutes on average vs 27 minutes for ST-Net, on CentOS 7 with a 24-core Intel Xeon 8260 CPU and one NVIDIA V100 (32GB) GPU.
- Reproducibility gaps
- The text says HisToGene can be downloaded and the HER2+ data come from a website, but both URLs are missing from the extracted text; cSCC data are under GEO accession GSE144240. cSCC results sit in Supplementary Note 1, not in the text.
Limitations (6)
- Like other deep learning models, HisToGene needs a relatively large training set. · stated by the authors
- On the small, platform-mixed cSCC dataset neither HisToGene nor ST-Net could predict expression with high accuracy. · stated by the authors
- ST-Net's released code is unmaintained, so the comparison uses the authors' own re-implementation of ST-Net. · stated by the authors
- ARI cannot be computed for the full super-resolution clusters, so their agreement with pathologist annotation is judged by eye. · stated by the authors
- Main evaluation rests on one HER2+ breast cancer cohort (32 sections, 7 patients), and the clustering comparison on only 6 annotated sections. · noted in this summary
- Even the best-predicted genes correlate only modestly with observed expression (mean R = 0.32 for GNAS, 0.26-0.27 for the next three), which tempers the claim of accurate prediction. · noted in this summary
Who should read:Read this if you want to predict spatial gene expression, including sub-spot detail, from H&E slides using spatial transcriptomics as training data, particularly in tumour tissue.
Reproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- Spatial Transcriptomics Prediction from Histology jointly through Transformer and Graph Neural Networks
Yuansong Zeng, Zhuoyi Wei, … Yuedong Yang·Briefings in BioinformaticsPeer-reviewed·Jul 17, 2022
- Generalization of deep learning models for predicting spatial gene expression profiles using histology images: A breast cancer case study
Yuanhao Jiang, Jacky Xie, … Quan Hoang Nguyen·bioRxiv·Sep 20, 2023
- Efficient prediction of a spatial transcriptomics profile better characterizes breast cancer tissue sections without costly experimentation
Taku Monjo, Masaru Koido, … Yoichiro Kamatani·Scientific ReportsPeer-reviewed·Mar 8, 2022
- Predicting Spatial Transcriptomics from H&E Image by Pretrained Contrastive Alignment Learning
Jiawei Zou, Kai Xiao, … Luonan Chen·bioRxiv·Jun 15, 2025
- Gene Expression Prediction from Histology Images via Hypergraph Neural Networks
Bo Li, Yong Zhang, … Qianqian Song·Briefings in BioinformaticsPeer-reviewed·Sep 23, 2024
- Benchmarking the translational potential of spatial gene expression prediction from histology
Adam S. Chan, Chuhan Wang, … Jean Yee Hwa Yang·bioRxiv·Dec 12, 2023