Finetuning Foundation Models for Temporal Clinical Transcriptomics Data

Abstract

BackgroundTimeseries clinical transcriptomic datasets offer the opportunity to gain insights into the dynamics of disease mechanisms/treatment responses. However, their utility in uncovering temporal patterns is often limited by high noise levels and small sample sizes. Leveraging foundational gene embeddings and incorporating interaction information can help address these challenges, improve gene network analysis, and enable the detection of subtle changes that drive disease progression or drug response. ResultsWe finetuned gene embeddings from foundation models using healthy tissue gene expression data and used them in temporal GNNs to model gene expression of responder and non-responders to treatment in 3 disease datasets - ulcerative colitis, Crohns disease and psoriasis. Application of our method to these datasets confirmed known mechanisms associated with drug action, and also identified key differences between activated and repressed pathways for responders and non responders including B-Cell activation and mitochondria related activity in ulcerative colitis patients. ConclusionFinetuning gene embeddings from foundation models provide a richer context to model gene expression data compared to using them in their naive state. Even with smaller sample sizes, results from GNN-based temporal models outperform traditional methods by detecting known mechanisms of response and unraveling role of genes and mechanisms not known to be associated with response and non-response. Code AvailabilityCode and data are available in a public GitHub repository - https://github.com/Sanofi-Public/GNN-Timeseries