Few-shot regulatory sequence design with a genomic language model

Abstract

A central challenge in generative biology is translating knowledge learned from natural sequences and experiments into designs with desired functions. In-context learning (ICL) is an emergent capability of large language models that allows them to adapt to new tasks from a few examples without additional training, enabling these models to become general-purpose learners. Here, we show that the genome language model, Evo 2, exhibits generative ICL in biological sequence design. Prompting with just tens of examples conditions Evo 2 to generate novel and diverse regulatory DNA sequences without any additional training. Mechanistic analysis suggests that genomic ICL may originate from induction, similar to phenomena observed in natural language. Reinforcement learning leveraging sequence-to-function models as experiment-derived rewards further strengthens ICL. We first validate this approach by generating enhancers that drive cell-type-specific gene expression in three human cell lines. Extending this approach in vivo, enhancers designed using human regulatory element data selectively label layer 5 extratelencephalic neurons in the mouse cortex with high specificity and stronger activity than existing enhancers. Our study provides evidence of emergent properties in genomic sequence models and introduces a promising approach for improving functional conditioning in generative biology.