Contrastive Fitness Learning: Reprogramming Protein Language Models for Low-N Learning of Protein Fitness Landscape

Abstract

Machine learning (ML) is revolutionizing our ability to model the fitness landscape of protein sequences, which is critical to answering fundamental life science questions and addressing important protein engineering applications, such as quantifying the pathogenicity of disease variants, forecasting viral evolution in a pandemic, and engineering new antibodies. Recently, the protein language model (pLM) has emerged as an effective ML tool in deciphering the intrinsic semantics of protein sequences and become the foundation of state-of-the-art ML solutions for many problems in protein biology. However, significant challenges remain in leveraging pLMs for protein fitness prediction, in part due to the disparity between the scarce number of sequences functionally characterized by high-throughput assays and the massive data samples required for training large pLMs. To bridge this gap, we introduce Contrastive Fitness Learning (ConFit), a pLM-based ML method for learning the protein fitness landscape with limited experimental fitness measurements as training data. We propose a novel contrastive learning strategy to fine-tune the pre-trained pLM, tailoring it to achieve protein-specific fitness prediction while avoiding overfitting, even when using a small number (low-N) of functionally assayed mutant sequences for supervised fine-tuning. Evaluated across over 30 benchmark datasets of protein fitness, ConFit consistently provided accurate fitness predictions and outperformed several competitive baseline methods. Further analysis revealed that ConFits capability of low-N learning enabled sample-efficient active learning for identifying high-fitness protein variants. Collectively, our work represents a novel strategy to harness the potential of pLMs to elucidate the protein sequence-function relationship. The source code of ConFit is available at https://github.com/luo-group/ConFit.