Inherent limitations of zero-shot mutation prediction in protein engineering

Abstract

Predicting mutation effects is critical for protein engineering. A commonly pursued strategy is zero-shot prediction with deep-learning models, whose performance is typically evaluated using aggregate benchmark scores that obscure multidimensional biological tradeoffs. Here, we evaluate sequence, structure, and hybrid foundation models across curated datasets to expose the practical limitations of zero-shot predictions. We demonstrate that model performance is highly variable and heavily influenced by experimental noise and competing phenotypic goals. Notably, predictions from competing models often correlate more strongly with one another than with experimental outcomes. While current models effectively filter deleterious mutations and enrich high-scoring variants for natural functions, they struggle to identify precise optimal mutations and consistently fail to predict mutations that drive new-to-nature functions. These models learn from evolutionary constraints that tend to oppose the extreme biophysical optimizations required in targeted engineering. Consequently, zero-shot scoring serves as a valuable initial filter but cannot substitute for task-specific prediction.