Tom Sercu

The paper uses language modeling to create a comprehensive representation of protein biology, enabling structure prediction and discovery of high-affinity protein interactions.

ESM3, a 98-billion-parameter multimodal protein language model tokenized across sequence, structure, and function, designs a bright green fluorescent protein at 58% sequence identity to known fluorescent proteins.

Protein language models scaled to 15B parameters predict atomic-resolution structures from sequence up to 60x faster than the state of the art, producing over 617 million metagenomic structures.

A modular high-level programming language paired with an energy-based protein language model generates de novo all-atom protein sequences and structures satisfying user-specified geometric, secondary structure, symmetry, and multimerization constraints.

A sequence-only protein language model, sampled under a structure-compatibility energy, produced designs of which 152 of 228 tested behaved as soluble monomers, some with no detectable UniRef90 match.

Adding 12 million AlphaFold2-predicted structures to training lifts native sequence recovery on held-out backbones to 51.6%, versus 42.2% for the best experimental-only model; likelihoods score mutation effects zero-shot.

A single pre-trained protein language model, ESM-1v, scores mutations zero-shot about as well as family-specific DeepSequence across 41 deep mutational scans, with no experimental labels or retraining.

The paper introduces a Neural Potts Model that predicts pairwise coupling matrices for protein families using amortized optimization, showing improved performance on low-depth multiple sequence alignments.

The paper trains a deep language model on 250 million protein sequences to learn biological representations that capture structural and functional properties.

One protein language model trained across many families on whole multiple sequence alignments, with interleaved row and column attention, reportedly outperforms unsupervised structure-learning methods by a wide margin.

Attention maps of the 650M-parameter protein language model ESM-1b, read out by a logistic regression fitted on 20 proteins, predict contacts better on average than Gremlin's alignment-based Potts models.