DSA-DeepFM: A Dual-Stage Attention-Enhanced DeepFM Model for Predicting Anticancer Synergistic Drug Combinations

Abstract

MotivationDrug combinations are crucial in combating drug resistance, reducing toxicity, and improving therapeutic outcomes in the treatment of complex diseases. As the number of available drugs grows, the potential combinations increase exponentially, making it impractical to rely solely on biological experiments to identify synergistic drug pairs. Consequently, machine learning methods are increasingly used to systematically screen for synergistic drug combinations. However, most current approaches prioritize predictive performance by integrating auxiliary information or increasing model complexity. By overlooking the biological mechanisms behind feature interactions, their effectiveness in predicting drug synergy can be limited. ResultsWe present DSA-DeepFM, a deep learning model that integrates a dual-stage attention (DSA) mechanism with Factorization Machines (FMs) to improve drug synergy prediction by addressing complex biological feature interactions. The model incorporates categorical and auxiliary numerical inputs, embedding them into high-dimensional spaces and then fusing them through the DSA mechanism to capture both field-aware and embedding-aware patterns. These patterns are then processed by a DeepFM module, which captures low-order and high-order feature interactions before making final predictions. Validation testing demonstrates that DSA-DeepFM significantly outperforms traditional machine learning and state-of-the-art deep learning models. Additionally, t-SNE visualizations confirm the models discriminative power at various stages. As a case study, we use our model to identify eight novel synergistic drug combinations, three of which are well-supported by existing wet-lab experiments, underscoring its practical utility and potential for future applications.