henhao
Sign in
Pathologynpj Digital Medicine
Peer-reviewed journal article

STPath: A Generative Foundation Model for Integrating Spatial Transcriptomics and Whole Slide Images

Tinglin HuangiD, Tianyu LiuiD, Mehrtash BabadiiD, Rex YingiD, Wengong JiniD

Abstract

Spatial transcriptomics (ST) has shown remarkable promise in pathology applications, shedding light on the spatial organization of gene expression and its relationship to the tumor microenvironment. However, its clinical adoption remains constrained due to the limited scalability of current sequencing technologies. While recent methods attempt to infer ST from whole slide images (WSIs) using pretrained image encoders, they remain restricted by limited gene coverage, organ-specific training, and the need for dataset-specific fine-tuning. In light of this, we introduce STPath, a generative foundation model pretrained on a large-scale collection of WSIs paired with ST profiles. This extensive pretraining enables STPath to directly predict gene expression across 38,984 genes and 17 organs without requiring downstream fine-tuning. STPath integrates multiple data modalities, including histological images, gene expressions, organ type, and sequencing technology information, within a novel geometry-aware Transformer architecture. Unlike previous methods that directly map WSIs to gene expression, STPath is trained using a masked gene expression prediction objective guided by tailored noise schedules, effectively balancing between capturing gene-gene dependencies and performing high-quality predictions. We evaluate STPath across 6 tasks spanning 23 datasets and 14 biomarkers, including gene expression prediction, spot imputation, spatial clustering, biomarker prediction, gene mutation prediction, and survival prediction. These results demonstrate STPaths strong ability to infer spatially resolved gene expression and reveal crucial pathological structures within tissue samples, underscoring its promise for scalable ST-based pathology applications.

PDFDOICodeData
Full-text digest ↓

02 /Specification & Classification

Click any tag to browse matching papers
Citations
27
FWCI
8.15
Field percentile
top 1%
OpenAlex Topics

In-depth summary

✦ Multi-agent verified full-text synthesis

Claude Opus 5.5 · xhigh effort · 7 Nov 2026 · from the full text of version 2 (18 May 2025) · CC BY 4.0

A masked-expression Transformer trained on 983 paired H&E and spatial transcriptomics slides predicts 38,984 genes from histology without fine-tuning, leading HEST-Bench and improving survival and mutation prediction.

Methods that infer spatial transcriptomics from whole-slide images (WSIs) usually fit a prediction head on pathology foundation-model features using small organ-specific datasets, each bound to one gene panel and sequencing technology, so they generalize poorly and need retraining for every deployment. STPath instead pretrains one generative model across organs, genes and technologies, and its masked objective lets measured spots be supplied as prompts at inference.

Key results

  1. ShownWithout fine-tuning, STPath gives the best average gene expression prediction on HEST-Bench. The comparison is against UNI, Gigapath, BLEEP and TRIPLEX, all retrained on the same pretraining corpus.

    Top 50 HVGs across 10 cancer datasets: first in PCC on 8/10 and in AMI on 9/10; average +6.9% PCC over UNI and +14.4% AMI over TRIPLEX (two-sided Wilcoxon P < 0.001). PCC 0.75 on IDC, 0.60 on SKCM, 0.559 on LUNG. Six cancer biomarkers on 4 datasets: +9.3% over the next-best model (P < 0.001); GATA3 and MYBPC1 PCC > 0.95. · 2.2 STPath accurately infers spatially-resolved…; Gene Expression Prediction; 2.4 STPath accurately predicts cancer-related…; Biomarker Prediction

  2. ShownSupplying measured expression for a small fraction of spots as prompts improves prediction on the remaining masked spots. The gain was consistent across the HEST-Bench datasets where zero-shot PCC was below 0.7.

    With 5% prompted spots: CCRCC +0.257 PCC (absolute), HCC 0.104 to 0.179, PRAD 0.298 to 0.386. From 0% to 20% prompting: LYMPH_IDC 0.070 to 0.293, READ 0.300 to 0.374, PAAD 0.471 to 0.490. Prompted spots grow outward from the leftmost spot to its nearest neighbours, and only masked spots are scored. · STPath effectively imputes gene expression with…; Spot Imputation; A.3 Full Results of In-context Prediction

  3. ShownLeiden clusters of STPath-predicted expression match pathologist annotations (brain layers, tumor regions) better than baseline predictions on all five annotated datasets (42 samples).

    Scored by AMI and homogeneity, averaged per dataset: STPath was best on every dataset, beating the strongest baseline by 0.121 on human brain and by 29.6% on the 10x sample. Wilcoxon rank-sum tests on predicted expression between annotated regions return known tumor markers, such as SPAG5, MMP11 and IGFBP3 in breast samples. · 2.3 Gene expressions predicted by STPath capture…; Spatial Clustering; 2.4 STPath accurately predicts cancer-related…; Biomarker Prediction

  4. ShownAdding STPath-predicted expression of 14 biomarkers to spot image features in an ABMIL head improves weakly supervised survival and mutation prediction, for both UNI and Gigapath features, in most cohorts.

    Survival (C-index; MBC, SURGEN, HNSC, LUAD): gains in 7 of 8 cases, on average +6.1% for UNI and +5.6% for Gigapath; on HNSC, UNI rose from 0.665 to 0.771 and Gigapath from 0.591 to 0.698. Mutation (AUC; 6 CPTAC cohorts): gains in 18 of 24 settings, +5.6% for UNI and +5.0% for Gigapath. 6/2/2 patient-level splits, 3 seeds. · 2.5 STPath enhances slide-level prediction via…; Weakly Supervised Survival Prediction; Weakly Supervised Gene Mutation Prediction; Weakly Supervised Classification

  5. SuggestedWith no training, the within-slide variance of predicted biomarker expression can serve as a slide-level mutation score, reaching AUC 0.73 for TP53 in breast cancer.

    No fine-tuning; all slides in each dataset evaluated: TP53 AUC 0.73 and PIK3CA 0.60 (BRCA), TP53 0.73 and KRAS 0.66 (COAD), EGFR 0.66 (LUAD); on average +13.9% over the best-performing baselines. · Unsupervised Gene Mutation Prediction; Unsupervised Gene Mutation Prediction

Methods, data and reproducibility

Each spot is a token that sums Gigapath image features, log1p expression over a 38,984-gene vocabulary, and organ and sequencing-technology embeddings. A 4-layer Transformer with an E(2)-invariant, frame-averaged spatial attention bias regresses masked expression (MSE), trained with Uniform(64, 1024) region sampling, Beta(10, 1) masking rates and an 80% HVG / 20% all-gene target schedule.

  • Pretraining corpus: paired H&E WSIs and spatial transcriptomics from HEST-1K and STImage-1k4m, deduplicated, with benchmark samples removed; Visium HD, Xenium, Visium and Spatial Transcriptomics · n = 983 WSIs (928 for pretraining); 38,984 genes; 17 organs
  • HEST-Bench gene expression benchmark of cancer datasets from different organs (e.g. IDC, SKCM, LUNG, CCRCC, HCC, PRAD, COAD, PAAD, READ, LYMPH_IDC) · human (patient-level splits) · n = 10 datasets
  • Human brain spatial transcriptomics samples with layer annotations · human · n = 12 samples
  • GSE213688 samples annotated with breast cancer subtypes · n = 14 samples · GSE213688
  • Chen (tumor annotations), Andersson (tumor cell types) and Erickson (prostate; benign/malignant, ISUP Gleason grade groups) annotated spatial transcriptomics samples · n = 1, 8 and 7 samples
  • CPTAC mutation cohorts BRCA, COAD, LUAD, GBM, LSCC, HNSC (labels via Patho-Bench; TP53, KRAS, PIK3CA, EGFR, STK11, ARID1A, KEAP1, CASP8) · human (patient-level splits) · n = 6 datasets
  • Survival cohorts MBC, SURGEN, HNSC and LUAD · human (patient-level splits) · n = 4 datasets
Methods & tools
STPath, Gigapath, UNI, BLEEP, TRIPLEX, ABMIL, Trident, Scanpy, Leiden clustering, PyTorch, HEST-Bench, Patho-Bench
Code
https://github.com/Graph-and-Geometric-Learning/STPath
Data & accessions
https://huggingface.co/datasets/MahmoodLab/hest-bench/tree/mainhttps://huggingface.co/datasets/MahmoodLab/Patho-Bench
Software & environment
PyTorch 2.3.1, Python 3.10.16
Assay conditions & compute
One Linux server: AMD EPYC 9534 64-core CPU, 1024G RAM, 8 NVIDIA L40S-48GB GPUs; training time not stated.
Reproducibility gaps
Pretrained weights are said to be on a public hub, but no link is given in the text; WSI download sources are listed in Table 3, not in the text; training time not reported; the pretraining slide count is 983 in one section and 986 in another.
Limitations (3)
  • Correlation stays low for some organs, such as CCRCC in HEST-Bench. The authors attribute this to scarce relevant pretraining data and propose adding more diverse datasets and modalities such as scRNA-seq. · stated by the authors
  • The benchmark replaces HEST-Bench's per-fold training with zero-shot averaging, and the authors retrained all baselines under STPath's own setup (loss, slide sampling, Gigapath encoder), so the numbers are not directly comparable with published HEST-Bench results. · noted in this summary
  • Slide-level gains come from random 6/2/2 patient-level splits averaged over 3 seeds, with no significance tests, and hold in 18 of 24 mutation settings. Unsupervised AUCs (0.60 to 0.73) are highlighted for five biomarker cases; the full set evaluated is not given in the text. · noted in this summary

Who should read:Read this if you want to predict spatial gene expression from H&E slides without per-dataset fine-tuning, or to add inferred spatial transcriptomics as features in slide-level survival or mutation models.

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISZeng '22Zhang '25Kaur '25Xiao '26Ahsan '26Pang '21
76% semantic matchBioinformatics
Yuansong Zeng, Zhuoyi Wei, Weijiang Yu+6 more·Briefings in Bioinformatics·Jul 17, 2022· 212 citations