Zeming Lin

Language Modeling Materializes a World Model of Protein Biology

The paper uses language modeling to create a comprehensive representation of protein biology, enabling structure prediction and discovery of high-affinity protein interactions.

Simulating 500 million years of evolution with a language model

ESM3, a 98-billion-parameter multimodal protein language model tokenized across sequence, structure, and function, designs a bright green fluorescent protein at 58% sequence identity to known fluorescent proteins.

Evolutionary-scale prediction of atomic level protein structure with a language model

Protein language models scaled to 15B parameters predict atomic-resolution structures from sequence up to 60x faster than the state of the art, producing over 617 million metagenomic structures.

A high-level programming language for generative protein design

A modular high-level programming language paired with an energy-based protein language model generates de novo all-atom protein sequences and structures satisfying user-specified geometric, secondary structure, symmetry, and multimerization constraints.

Learning inverse folding from millions of predicted structures

Adding 12 million AlphaFold2-predicted structures to training lifts native sequence recovery on held-out backbones to 51.6%, versus 42.2% for the best experimental-only model; likelihoods score mutation effects zero-shot.

Neural Potts Model

The paper introduces a Neural Potts Model that predicts pairwise coupling matrices for protein families using amortized optimization, showing improved performance on low-depth multiple sequence alignments.

Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences

The paper trains a deep language model on 250 million protein sequences to learn biological representations that capture structural and functional properties.