henhaobio
Sign in

bioRxivBioinformaticsPreprint

PIE: Generalizing perturbation effects across unseen perturbations, contexts and datasets

Rishi VermaiD, Abhinav AdduriiD, Beatrice BevilacquaiD, Basak EraslaniD, Dave BurkeiD, Hani GoodarziiD, Yusuf RoohaniiD

PDFDOICodeData
In-depth full-text digest available · Jump to digest ↓

Abstract

Predicting cellular response to perturbations is key to understanding biological mechanisms and selecting therapeutic targets. However, generalizing across cellular contexts, perturbations, and experimental datasets remains challenging due to incomplete representations of the system being perturbed and the difficulty of measuring true perturbation effects within that system. We introduce PIE, a model that addresses these limitations by reformulating the learning task around population level perturbation effects and incorporating auxiliary inputs that better characterize the biological system. PIE directly predicts differentially expressed genes and changes in gene expression for each (context, perturbation) pair, using external biological knowledge, baseline gene expression, and observed perturbation responses. Its architecture accommodates input sources with variable feature sets and predicts effects on genes seen and unseen during training. On the Replogle Nadig dataset, PIE achieves state of the art performance on most metrics across all tested generalization settings, including the most challenging setting where both the context and perturbation are unseen during training. For predicting differentially expressed genes, PIE achieves 1.2 to 3.2 times the AUPRC of the strongest baseline in each setting: across unseen contexts, perturbations, experimental datasets, and jointly unseen contexts and perturbations. Overall, PIE provides a framework for generalizing perturbation effects across diverse experimental conditions.

In-depth summary

Claude Opus 5.5 · xhigh effort · 7 Nov 2026 · from the full text of version 1 (5 Oct 2026) · CC BY 4.0

Predicting differentially expressed genes and expression shifts per condition from database embeddings plus per-gene evidence, PIE reaches 1.2–3.2 times the best baseline's AUPRC on unseen contexts, perturbations and datasets.

Generalizing perturbation-response predictions across contexts, perturbations and datasets is hindered by thin representations of the perturbed system and by single-cell readouts that cannot directly measure a cell's perturbation effect. Unlike most prior models, which rely mainly on transcriptomes and predict cell-level states, PIE learns condition-level effects from external database knowledge plus measured evidence, and one architecture handles unseen contexts, perturbations, datasets and genes.

Key results

  1. ShownWith a whole cell line held out (perturbations seen), PIE leads on five of six metrics on Replogle–Nadig. It beats Rhaister-O, the strongest baseline, at ranking differentially expressed (DE) genes and at correlating log fold changes (LFC), but trails it slightly on effect-size correlation.

    Mean over 4 folds: AUPRC (area under the precision-recall curve for DE genes) 0.424 vs 0.344 for Rhaister-O; Spearman LFC 0.664 vs 0.591; Spearman effect size 0.734 vs 0.741 for Rhaister-O. Other comparators were State, Stack, TabICL, perturbation mean and context mean. · 4.2. Generalization within dataset; Baselines

  2. ShownFor unseen genetic perturbations in seen cell lines, PIE leads on all six metrics. When the cell line and the perturbation are both unseen, it beats context mean, the only applicable baseline, on every metric.

    Unseen perturbations: AUPRC 0.304 vs 0.094 for TabICL (stated as more than threefold); COMPASS-N and PRESAGE were also compared. Both unseen: AUPRC 0.239 vs 0.088 and Discrimination Score L1 0.643 vs 0.502 (about random) for context mean, whose Spearman metrics are undefined. · 4.2. Generalization within dataset; Baselines

  3. ShownTrained only on four other datasets (X-Atlas/Orion, Tahoe-100M, Arc VCC 2025 and Jiang et al.), with shared and derived cell lines removed, PIE scores best on five of six metrics for seen perturbations in Replogle–Nadig. It still carries signal for perturbations absent from all training data.

    Mean over 4 held-out cell lines. Seen perturbations: AUPRC 0.378 vs 0.283 (Rhaister-O); Spearman effect size 0.627 vs 0.547; DE direction match 0.727 vs 0.733 for perturbation mean. Unseen perturbations vs context mean: AUPRC 0.252 vs 0.102; Jaccard 0.123 vs 0.077. Highest AUPRC also reported when other datasets are held out (appendix tables). · 4.3. Generalization across datasets; B.4. Generalization across datasets; Replogle–Nadig held out

  4. ShownExact Shapley values over five input groups show that, for unseen cell lines, almost all of the AUPRC gain comes from two groups: responses to the same perturbation in other contexts, and the perturbation knowledge sources. For unseen perturbations, the knowledge sources mainly help tell perturbations apart.

    Unseen contexts: AUPRC 0.238 with no groups vs 0.422 with all; Same-pert Evidence plus Perturbation Knowledge Sources give 0.169 of the 0.184 gain. Unseen perturbations: AUPRC gain 0.045 spread over all groups; knowledge sources give 0.205 of the 0.207 Discrimination Score L1 gain. Both unseen: AUPRC gain only 0.010. · 4.4. Contributions of input signals; Shapley values; Models and training

  5. ShownOf the five perturbation knowledge sources, DepMap knockout-effect profiles add the most AUPRC for unseen perturbations. ESM2 adds none and the gene text description is slightly negative, yet every source raises Discrimination Score L1 in that regime.

    Unseen perturbations: sources lift AUPRC from 0.281 to 0.306; DepMap 0.013, STRING 0.008, NCBI Gene Annotations 0.006, ESM2 0.000, Perturbation Description −0.002. Discrimination Score L1 contributions run from 0.020 (ESM2) to 0.066 (DepMap), 0.206 together. The Perturbation Description's AUPRC attribution is below zero in every regime. · Results; Ablation of Perturbation Knowledge Sources

Methods, data and reproducibility

PIE is a Perceiver IO transformer. Learned latents cross-attend to embedded knowledge-source tokens for the context and the perturbation. Each gene is then decoded by heads for a DE call, a signed log2 fold change and a binned pseudobulk delta expression; its query is a text embedding of its NCBI Gene entry plus per-gene evidence (control mean; same-perturbation and same-context response statistics).

  • Replogle–Nadig CRISPRi Perturb-seq essential-gene screens, single-gene knockdowns in four cell lines; main benchmark · human (cell lines K562, hTERT-RPE1, HepG2, Jurkat) · n = 5,459 conditions over 4 contexts; 611,211 cells; 6,642 genes · GEO GSE264667 (other releases on figshare)
  • Tahoe-100M single-cell small-molecule screen: 379 compounds at 0.05, 0.5 and 5 μM in 50 cancer cell lines with DMSO controls · n = 56,827 conditions; 95.6M cells kept; 18,151 genes
  • Jiang et al. (2025) CRISPRi screen of 218 signalling regulators in 6 cell lines under 5 cytokine stimuli (30 contexts) · human (A549, BxPC-3, HAP1, HT-29, K562, MCF7 cell lines) · n = 461 of 1,626 conditions kept after knockdown filter · GEO GSE281048
  • X-Atlas/Orion genome-wide dual-guide CRISPRi screens · human (HCT116 and HEK293T cell lines) · n = 13,284 conditions; 37,443 genes · doi:10.25452/figshare.plus.29190726
  • Arc VCC 2025 (2025 Virtual Cell Challenge) H1 CRISPRi data · human (H1 hESC) · n = 190 conditions; 18,151 genes
  • L1000 compound transcriptional activity scores used as a chemical knowledge source · n = 83 cell lines · GEO GSE92742; GSE70138
Methods & tools
Perceiver IO, text-embedding-3-large, Qwen3.5-35B-A3B-Base, ESM2 (esm2_t33_650M_UR50D), STRING v12.0 (SPACE embedding), ChemBERTa-77M-MTR, DepMap Chronos 25Q3; L1000; PRISM; JUMP Cell Painting, Cellosaurus; NCBI Gene; PubChem; UniProt, gpudge, cell-load, cell-eval / cell-eval2, Baselines: State, Stack, Rhaister-O, COMPASS-N, PRESAGE, TabICL
Software & environment
Embedding resources are versioned: ESM2 esm2_t33_650M_UR50D, ChemBERTa-77M-MTR, STRING v12.0 SPACE, DepMap 25Q3, text-embedding-3-large, Qwen3.5-35B-A3B-Base. Software package versions and containers are not stated.
Reproducibility gaps
Hardware, run time and memory are not reported, and each ablation model was trained with a single seed. Per the AI use statement, AI coding agents wrote all project code, with the authors directing, supervising and reviewing it.
Limitations (6)
  • Performance falls when the context and perturbation are both unseen; in the Replogle–Nadig cross-dataset test, every metric drops for perturbations absent from all training datasets. · stated by the authors
  • Predictions are weaker for perturbations with small effects, where it is hardest to tell biological signal from experimental variation. · stated by the authors
  • Knowledge-source coverage is uneven, especially for the measured compound profiles (L1000, PRISM, JUMP). Missing entries contribute no tokens or are set to zero. · stated by the authors
  • Perturbation knowledge sources improve direction, LFC and discrimination without necessarily improving DE ranking; their mean AUPRC attribution is negative when both entities are unseen. · stated by the authors
  • Comparators in the hardest regimes are weak. Context mean (a zero-shift predictor for unseen contexts) is the only baseline when both entities are unseen, and Rhaister-O runs its full model on about a third of Replogle–Nadig test conditions and only its fallback in the cross-dataset tests. · noted in this summary
  • The evaluation pipeline is the authors' own: cell-eval metrics were adapted to probability outputs, cell-generating baselines were converted to pseudo-probabilities via gpudge, and Stack was retrained from scratch. Baseline scores therefore depend on these conversions. · noted in this summary

Read this if you predict transcriptional responses to CRISPRi knockdowns or compounds in cell lines, perturbations or whole datasets absent from training, or want to know which external knowledge sources actually help.

Citations
1

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISLittman '25Chen '24Yu '25Molina '26Su '26Li '25
64% semantic matchBioinformatics
Russell Littman, Jacob Levine, Sepideh Maleki+11 more·bioRxiv·Jun 3, 2025· 7 citations