Predicting Methylation from Sequence and Gene Expression Using Deep Learning with Attention

Abstract

DNA methylation has been extensively linked to alterations in gene expression, playing a key role in the manifestation of multiple diseases, most notably cancer. For this reason, researchers have long been measuring DNA methylation in living organisms. The relationship between methylation and expression, and between methylation in different genomic regions is of great theoretical interest from a molecular biology perspective. Therefore, several models have been suggested to support the prediction of methylation status in samples. These models, however, have two main limitations: (a) they heavily rely on partially measured methylation levels as input, somewhat defeating the object as one is required to collect measurements from the sample of interest before applying the model; and (b) they are largely based on human mediated feature engineering, thus preventing the model from unveiling its own representations. To address these limitations we used deep learning, with an attention mechanism, to produce a general model that predicts DNA methylation for a given sample in any CpG position based solely on the sample's gene expression profile and the sequence surrounding the CpG.\n\nWe show that our model is capable of generalizing to a completely separate test set of CpG positions and subjects. Depending on gene-CpG proximity conditions, our model can attain a Spearman correlation of up to 0.8 and MAE of 0.14 for thousands of CpG sites in the test data. We also identify and analyze several motifs and genes that our model suggests may be linked to methylation activity, such as Nodal and Hand1. Moreover, our approach, and most notably the use of attention mechanisms, offers a novel framework with which to extract valuable insights from gene expression data when combined with sequence information.\n\nThe code and trained models are available at: https://github.com/YakhiniGroup/Methylation