Predicting and Interpreting Protein Developability via Transfer of Convolutional Sequence Representation

Abstract

Engineered proteins have emerged as novel diagnostics, therapeutics, and catalysts. Often, poor protein developability - quantified by expression, solubility, and stability - hinders utility. The ability to predict protein developability from amino acid sequence would reduce the experimental burden when selecting candidates. Recent advances in screening technologies enabled a high-throughput developability dataset for 105 of 1020 possible variants of protein ligand scaffold Gp2. In this work, we evaluate the ability of neural networks to learn a developability representation from a high-throughput dataset and transfer this knowledge to predict recombinant expression beyond observed sequences. The model convolves learned amino acid properties to predict expression levels 44% closer to the experimental variance compared to a non-embedded control. Analysis of learned amino acid embeddings highlights the uniqueness of cysteine, the importance of hydrophobicity and charge, and the unimportance of aromaticity, when aiming to improve the developability of small proteins. We identify clusters of similar sequences with increased developability through nonlinear dimensionality reduction and we explore the inferred developability landscape via nested sampling. The analysis enables the first direct visualization of the fitness landscape and highlights the existence of evolutionary bottlenecks in sequence space giving rise to competing subpopulations of sequences with different developability. The work advances applied protein engineering efforts by predicting and interpreting protein scaffold developability from a limited dataset. Furthermore, our statistical mechanical treatment of the problem advances foundational efforts to characterize the structure of the protein fitness landscape and the amino acid characteristics that influence protein developability. Significance statementProtein developability prediction and understanding constitutes a critical limiting step in biologic discovery and engineering due to limited experimental throughput. We demonstrate the ability of a machine learning model to learn sequence-developability relationships first through the use of high-throughput assay data, followed by the transfer of the learned developability representation to predict the true metric of interest, recombinant yield in bacterial production. Model performance is 44% better than a model not pre-trained using the high-throughput assays. Analysis of model behavior reveals the importance of cysteine, charge, and hydrophobicity to developability, as well as of an evolutionary bottleneck that greatly limited sequence diversity above 1.3 mg/L yield. Experimental characterization of model predicted candidates confirms the benefit of this transfer learning and in-silico evolution approach.