Emergence of Biological Structural Discovery in General-Purpose Language Models

Abstract

Large language models (LLMs) are evolving into engines for scientific discovery, yet the assumption that biological understanding requires domain-specific pre-training remains largely unchallenged. Here we report that general-purpose LLMs possess an emergent capability for biological structural discovery. Under strict, shortcut-controlled evaluation, a small-scale GPT-2 (124M) fine-tuned solely on English paraphrase discrimination detects protein homology zero-shot at ROC-AUC 0.79 on a shortcut-controlled benchmark. Controls establish that the ability is conferred by pre-training, not architecture: a randomly initialized GPT-2 is at chance (0.52). To exclude the possibility that public checkpoints were contaminated with biological data, we train our own GPT-2 from scratch on an English-only web corpus; it reproduces the transfer (0.76), proving the effect arises from linguistic pre-training alone. Network-based interpretability reveals a deep structural isomorphism: the discriminative signal localizes to deep layers (0.97 at layer 9), and attention analysis surfaces modality-agnostic "difference" operators. Scaling to massive instruction-tuned models further improves performance, including in the remote-homology "twilight zone", which we report as an exploratory upper bound because those models training corpora are undisclosed. We formalize these tasks through the BioPAWS benchmark. Our controlled results--obtained entirely on models with known training data--establish that abstract logical structures distilled from human language constitute a genuine, if bounded, cognitive prior for decoding the syntax of biology.