Generating functional protein variants with variational autoencoders

Abstract

The design of novel proteins with specified function and controllable biochemical properties is a longstanding goal in bio-engineering with potential applications across medicine and nanotechnology. The vast expansion of protein sequence databases over the last decades provides an opportunity for new approaches which seek to learn the sequence-function relationship directly from natural sequence variation. Advances in deep generative models have led to the successful modelling of diverse kinds of high-dimensional data, from images to molecules, allowing the generation of novel, realistic samples. While deep models trained on protein sequence data have been shown to learn biologically meaningful representations helpful for a variety of downstream tasks, their potential for direct use in protein engineering remains largely unexplored. Here we show that variational autoencoders trained on a dataset of almost 70000 luciferase-like oxidoreductases can be used to generate novel, functional variants of the luxA bacterial luciferase. We propose separate VAE models to work with aligned sequence input (MSA VAE) and raw sequence input (AR-VAE), and offer evidence that while both are able to reproduce patterns of amino acid usage characteristic of the family, the MSA VAE is better able to capture long-distance dependencies reflecting the influence of 3D structure. To validate the practical utility of the models, we used them to generate variants of luxA whose function was tested experimentally. As further evidence of the practicality of these methods for design, we showed that conditional variants of both models could be used to increase the solubility of luxA without disrupting function. Altogether 18/24 of the variants generated using the AR-VAE and 21/23 variants generated using the MSA VAE retained some luminescence activity, despite containing as many as 35 differences relative to any training set sequence. These results demonstrate the feasibility of using deep generative models to explore the space of possible protein sequences and generate useful variants, providing a method complementary to rational design and directed evolution approaches.