Transfer Learning and Permutation-Invariance improving Predicting Genome-wide, Cell-Specific and Directional Interventions Effects of Complex Systems

Abstract

With the advent of precision medicine, single-drug treatments may not fully satisfy the pursuit of precision medicine. However, single-drug treatments have accumulated a large amount of data and a considerable number of deep learning models. In this context, using transfer learning to effectively leverage the existing vast amount of single-compound intervention effect data to build models that can accurately predict the intervention effects of complex systems is highly worth investigating. In this study, we used a deep model based on permutation-invariance as the core module, pre-trained on a large amount of single-compound intervention data in cell lines, and fine-tuned on a small amount of complex system (like natural products) intervention data in cell lines, resulting in a predictive model named SETComp (the Concat version with ~200M parameters and the Add version with ~173M parameters). The two versions of SETComp achieved an accuracy of 93.86% and 92.70%, respectively, on the complex system-cell-gene association test set, improving by 5.82% to 27.59% compared to the baseline. When predicting the intervention effects of those complex systems the model had never encountered before, the accuracy increased by up to 24.83% compared to the baseline. In our in vitro validation, up to 88.65% of the predictions were confirmed to be correct, and the models output showed a significant positive correlation with the real-world foldchange. We further observed SETComps potential in various biomedical scenarios, achieving good performance in applications such as mechanism uncovering, repositioning, and compound synergy discovery.