DrugPT: A Flexible Framework for Integrating Gene and Chemical Representations in Perturbation Modeling

Abstract

Accurately modeling the transcriptional response of cells to drug perturbations is critical for drug discovery and precision medicine. Here, we propose DrugPT, a novel Transformer-based framework for predicting gene expression changes upon drug treatment. DrugPT consists of three modular components: (1) a gene representation module that encodes pre-treatment expression profiles using GPT-derived embeddings; (2) a drug representation module that captures chemical structure information using a pre-trained GPT2 model; and (3) a Transformer-based prediction module that integrates gene and drug embeddings to predict perturbed gene expression profiles. Unlike previous approaches, DrugPT formulates the prediction task as a sequence modeling problem, leveraging pre-trained language models for both gene and drug modalities. We demonstrate the architecture of DrugPT and highlight its generalizable design for encoding heterogeneous biological and chemical information. This work introduces a unified modeling framework, laying the foundation for future developments in multimodal perturbation prediction tasks.