Modelling antibody structures at the speed of language

Abstract

Protein structure prediction is currently substantially slower than obtaining sequence representations of proteins. This leads to most property prediction methods relying solely on trivial or learned sequence embeddings. However, contemporary structure prediction and sequence models are both based on Transformers, and structure prediction models often have fewer parameters, suggesting that there might be domains where accurate structure prediction adds no practical overhead to sequence-only modelling. Here, we demonstrate this can be achieved for adaptive immune proteins by introducing FlashABB, which predicts highly accurate antibody structures, and does so faster than even modestly-sized language models can embed sequences. As a component of FlashABB, we develop Flashpoint Attention, a fast and linear memory analog of Invariant Point Attention. To our knowledge, FlashABB is the first example of a model that accurately predicts protein structure faster than protein language models can generate embeddings, enabling efficient access to 3D information without the need for precomputed structures. Using FlashABB, we develop methods for predicting antibody stability and developability which can be scaled to repertoires of millions of sequences. Our results show how the computational bottleneck of protein structure prediction can be removed in some real-world cases. The code and model weights for FlashABB are available on GitHub: https://github.com/oxpig/FlashABB