Long-context Protein Language Modeling Using Bidirectional Mamba with Shared Projection Layers
Yingheng Wang, Zichen Wang, Gil Sadeh, Luca Zancato, Alessandro Achille, George Karypis, Huzefa Rangwala
This paper introduces LC-PLM, a protein language model based on BiMamba-S architecture, showing improved performance on protein tasks compared to Transformer-based models.
bioRxiv · Oct 29, 2024 · PreprintSynthetic Biology