Decoding Prokaryotic Whole Genomes with a Product-Contextualized Large Language Model

Abstract

Genomes encode the instructions for life, yet their interpretation requires models capable of capturing long-range genome-wide functional context at scale. Existing genomic foundation models have primarily focused on sequence-level representations, providing powerful tools for biological prediction while leaving complementary opportunities for function-level genome modeling. Here, we present GenSyntax, a function-level genome representation learning framework that models prokaryotic genomes as ordered sequences of gene-product descriptors. By treating gene products as semantic units, GenSyntax transforms annotated replicons into interpretable "genetic paragraphs" that capture genome-wide functional context. We trained GenSyntax on 49,250 annotated prokaryotic genomes and evaluated its ability across genome-scale tasks, including plasmid host prediction, gene-product disambiguation, genome contig ordering and gene essentiality prediction. Compared with general-purpose large language models and representative genomic foundation models, GenSyntax achieved competitive performance across multiple benchmarks, with independent external evaluations supporting its generalization beyond the original training datasets. GenSyntax embeddings also captured signals associated with microbial phenotypes, and gene essentiality predictions enabled exploratory design of minimal genomes. Together with the lightweight GenSyntax-Tiny model, GenSyntax provides a complementary, post-annotation framework for function-level analysis of prokaryotic genomes.