henhaobio
Sign in

bioRxivMolecular BiologyPreprint

An RNA Language Model trained on sequence alone reveals the structural logic of Internal Ribosome Entry Sites

Adam SychlaiD, Pierre BongrandiD, Grant YangiD, Anoop KrishnaniD, Jacob RulisoniD, R. Alexander Wesselhoeft, Namita BisariaiD, Silvi RouskiniD

PDFDOICodeData
In-depth full-text digest available · Jump to digest ↓

Abstract

Millions of RNA sequences are readily available, but the structures that determine their function are not. Picornaviruses initiate translation through Internal Ribosome Entry Sites (IRESes), RNA elements that recruit ribosomes independent of the 5` cap. These elements are large and highly divergent, and structural understanding remains limited to a handful of cases. We address this bottleneck by introducing an RNA language model (Albatross), trained purely on sequence, that predicts high-quality IRES structures at scale. We collect in cellulo chemical probing data for 96 full-length IRESes from divergent viruses and show that Albatross achieves far higher precision (0.80) than state-of-the-art predictions (0.47). Analyzing 75,000 IRES structures, we discover a novel Type II structural subclass and validate it experimentally. We demonstrate the pipeline broadly generalizes to identify functional structures, including tertiary contacts and alternative riboswitch structures. These findings establish Albatross as a scalable framework that accelerates RNA structure discovery and antiviral targeting.

In-depth summary

Claude Opus 5.5 · xhigh effort · 8 Nov 2026 · from the full text of version 2 (24 Sept 2026) · CC BY 4.0

Continued pretraining of an RNA language model on picornaviral 5′ UTRs alone predicts IRES base pairs with median precision 0.80 versus 0.47 for Infernal, and reveals a Type II subclass.

Full-length IRES structures are known for only a handful of cases, and thermodynamic folding needs experimental constraints on long RNAs while covariation analysis needs many related sequences that divergent IRESes lack. Albatross needs only sequence and no family alignment, and the paper pairs it with a 96-IRES probing and translation dataset.

Key results

  1. ShownAlbatross (RiNALMo continued-pretrained on IRES sequences, read out by dependency mapping) predicts base pairs with much higher precision than Infernal, eFold and RNAstructure, at comparable recall to Infernal and eFold. Using its pairs as hard constraints on RNAstructure gave the best F1.

    Median precision 0.80 vs 0.47 (Infernal), 0.44 (eFold), 0.47 (RNAstructure) on 95 IRESes with in cellulo DMS-MaPseq-constrained reference structures. Albatross-constrained RNAstructure reached median F1 0.64 vs 0.55 unconstrained. Albatross alone: median F1 0.60, recall 0.48. · Albatross predicts diverse IRES structures and…; Albatross achieves dramatically higher base-pair…; Selected parameters and performance (50k, epoch…

  2. ShownA polysome-profiling activity atlas of 96 full-length IRESes in six cell lines shows Type V IRESes as the most active and Type IV as the least. Most IRESes show significant cell-type-specific activity, and EMCV ranks poorly.

    In HEK293T the top ten IRESes were all Type V and EMCV ranked 87th of 96. Dual-luciferase on six selected IRESes kept the rank order (Caprine kobuvirus 12Q108 vs EMCV: 2.07 ± 0.25 and 2.26 ± 0.25 fold). 77/96 showed significant tropism, 38 of them moderate by Gini index; n = 4 replicates per cell type. · Type V IRES activity doubles that of standard…; IRESes exhibit moderate tissue-specific activity; Figure legends

  3. ShownThe model does not simply apply thermodynamics. Its calls depend on sequence context and learned motifs, and the pairs it recovers are the subset associated with IRES activity.

    A synthetic 12-bp hairpin absent from training gave no signal in the natively unstructured A71 region but was predicted in the native structured context. Mutating the Gallivirus GNRA tetraloop degraded the elbow signal. Albatross-recovered pairs tracked ribosomal loading (R^2 = 0.17); missed pairs did not (R^2 = 0.00). · Albatross uses sequence context to complement…; Thermodynamic stability is not sufficient for…; Albatross predicts a subset of base pairs…

  4. SuggestedMaps for ~75,000 IRESes recover types without labels and expose a Type II subclass with an extended AAG stem. DMS-MaPseq on one example (Sicinivirus MW684796) supports the predicted fold; in chicken DF1 cells stem mutants were indistinguishable from the no-IRES control, implying the stem matters for its low activity.

    ARI 0.78 vs manual type labels. The subclass cluster has 107 sequences. For Sicinivirus MW684796, DMS agreement with the predicted structure gave AUC-ROC 0.926. All stem mutants (Estem, α, Δ) were indistinguishable from the no-IRES control in chicken DF1 cells, where the wild type had low activity. · Dependency mapping at scale identifies a novel…; Figure legends

  5. SuggestedLarger base models transfer better across IRES types. The same continued-pretraining recipe recovered pseudo-circularization in hantavirus UTRs and apo/holo alternative pairings plus a kissing-loop contact in bacterial riboswitches.

    Models of 33M, 150M, 650M parameters trained on Type I IRESes only: F1 rose with size on Type I, II, V references (85 sequences). Trained on 5,041 hantavirus and 76,829 riboswitch sequences. Hantavirus and riboswitch results are shown as qualitative dependency maps (hantavirus: one outbreak strain; riboswitch: B. subtilis adenine plus other adenine and glycine examples) without a quantitative benchmark. · Dependency mapping learns generalizable…; Continued pretraining of RNA language models…; Generalization

Methods, data and reproducibility

RiNALMo (650M parameters) was continued-pretrained with masked-token prediction on about 50,000 diverse picornaviral 5′ UTR sequences. Single-point-mutation dependency maps were filtered into nested structures with thresholds tuned by 5-fold cross-validation, then benchmarked against in cellulo DMS-MaPseq structures of 95 IRESes.

  • Library of 96 full-length IRESes (365 to 1431 nt) as barcoded linear and circular RNA; DMS-MaPseq structures (95 usable in HEK293T) and polysome profiling in six cell lines · Picornavirus IRESes; human and mouse cell lines · n = 96 IRESes; 95 with structures; n = 4 per cell type · GEO: GSE34725 (DMS-MaPseq); GSE347508 (polysome profiling)
  • Picornaviral 5′ UTR training sets from NCBI Virus plus BLAST expansion (cd-hit clustered at 70% similarity) · Picornaviruses · n = 193,643 / 509,944 / 52,129 sequences (200k/500k/50k sets)
  • Dependency maps and structures for deduplicated IRES set used in family-wide analysis · Picornaviruses · n = 75,229 root sequences
  • Sicinivirus MW684796 IRES: DMS-MaPseq and dual-luciferase assays of stem mutants in chicken DF1 cells · Sicinivirus IRES; chicken cell line (Gallus gallus)
  • Orthohantavirus concatenated 5′+3′ UTRs for continued pretraining · Orthohantavirus · n = 5,041 sequences (from 15,375 records)
  • Bacterial riboswitch Rfam full-region sequences extended 150 nt downstream · Bacteria · n = 76,829 sequences, 10 ligand classes · Rfam RF00050, RF00059, RF00162, RF00167, RF00168, RF00174, RF00504, RF00522, RF01051, RF01734
Methods & tools
RiNALMo, Albatross, DMS-MaPseq, SEISMIC v24.3, Infernal 1.1.5, R-scape 2.6.4 (CaCoFold), eFold, RNAstructure 6.4, BLAST, cd-hit, polysome profiling, NCBI Virus
Software & environment
Python 3.11.5, numpy 2.2.6, scikit-posthocs 0.11.4, scipy 1.17.1 for statistics; SEISMIC v24.3; Infernal 1.1.5; R-scape 2.6.4; RNAstructure 6.4. Training: AdamW, learning rate 5e-6, batch size 8, seed 42.
Assay conditions & compute
Single L40 GPU per experiment; the 50k-sequence run took about twelve hours. Dependency map needs 3N + 1 forward passes per sequence of length N.
Reproducibility gaps
Plasmids available on request. The GitHub URL as printed in the text may be truncated. Other information is available from the lead contact on request.
Limitations (4)
  • Only picornaviral IRESes were studied. The ground-truth set contains no Type III IRESes, so performance on that type is untested. · stated by the authors
  • High precision comes with lower recall (median 0.48), so some substructures are missed in any given prediction. · stated by the authors
  • The Sicinivirus MW684796 IRES was tested only in a chicken cell line, where it showed weak activity; other cell types are needed to assess the extended stem's function. · stated by the authors
  • Reference structures are DMS-constrained models rather than independent structures, and the filter thresholds were tuned by cross-validation on the same 95 sequences. The hantavirus and riboswitch generalization is shown only as a few qualitative maps with no quantitative benchmark. · noted in this summary

Read this if you need family-scale RNA secondary structure predictions without alignments, or want full-length IRES activity data across six cell types.

Citations
1

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISPonti '25Velazquez '25Segar '26Das '25Sherlock '25Zhang '25
68% semantic matchBiochemistry
Riccardo Delli Ponti, Andrea Vandelli, Laura Broglia+1 more·Biology Direct·Dec 8, 2025· 1 citations