A generative language model decodes contextual constraints on codon choice for mRNA design

Abstract

Synonymous codon choice affects mRNA fate and protein output, posing a challenge for mRNA technology. Design of therapeutic mRNAs requires a model that captures biological nuance from natural sequences while identifying constraints relevant to synthetic expression. We present Trias, a generative encoder-decoder language model trained on millions of vertebrate coding sequences that successfully designs mRNAs with high protein output. Trias learns codon usage rules directly from sequence data, integrating local and global dependencies to generate species-specific codon sequences that reflect biological constraints. Without explicit training on protein expression, its zero-shot predictions correlate strongly with experimental measurements of mRNA stability and protein output. Moreover, we show experimentally that novel sequences generated by Trias for a therapeutically relevant target produce more protein than existing methods, demonstrating that preferences learned from natural sequences can lead to high output from synthetic sequences. Released open source with full code and weights, Trias provides a reproducible framework for synthetic mRNA design that reveals molecular and evolutionary principles behind codon choice.