Sampling Protein Language Models for Functional Protein Design

Abstract

Protein language models have emerged as powerful tools for learning rich protein representations, improving performance in tasks like structure prediction, mutation effect estimation, and homology detection. Their ability to model complex sequence distributions also holds promise for designing novel, functional proteins with broad applications in therapeutics, materials, and sustainability. However, due to the vastness of sequence space, efficient exploration methods are essential for protein engineering. Despite this, most existing protein design approaches using protein language models rely on single-mutant sampling strategies borrowed from Natural Language Processing, which fail to capture critical epistatic interactions between amino acid positions that are essential for protein function. In this work, we develop a comprehensive in silico protein design evaluation framework to systematically compare different sampling methods. After a thorough review of existing sampling strategies for language models, we introduce several approaches specifically tailored for protein design. We demonstrate that sampling strategies that consider multiple mutations simultaneously significantly outperform single-mutant approaches by better capturing epistatic effects between residue pairs. We evaluated these strategies using our framework, investigating the effects of key hyperparameters and providing practical guidance on the relative strengths of each method depending on design objectives.