Machine Learning in Bioinformatics

GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Wrap a genomic foundation model's inference methods once to score it on supervised and zero-shot GUE, VariantBenchmarks, BEND, TraitGym and ClinVar tasks; code is promised upon acceptance.

BioReason-Pro: Advancing Protein Function Prediction with Multimodal Biological Reasoning

A reasoning LLM combining ESM3 embeddings with GO-GPT term predictions produces protein function annotations that experts rated tie-or-better than UniProt in 79% of cases (SFT model).

Medium-sized protein language models perform well at transfer learning on realistic datasets

Medium-sized protein language models (e.g. ESM C 600M) with mean embeddings nearly match ESM-2 15B and ESM C 6B in transfer learning on mutational scanning and PISCES data.

Scaling unlocks broader generation and deeper functional understanding of proteins

Scaling sparse Mixture-of-Experts protein language models up to 46B parameters on 1.5T curated tokens broadens viable protein generation across families and amplifies gains from laboratory preference alignment.

SaProt: Protein Language Modeling with Structure-aware Vocabulary

Pairing each residue with its Foldseek 3D-structure token and pre-training an ESM-2-sized model on about 40 million AlphaFold2 structures beat ESM-2 and structure-based baselines on variant-effect and function tasks.

xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein

A 100-billion-parameter protein language model trained with masked and autoregressive objectives together, topping prior results on 15 of 18 benchmarks and powering a fast structure predictor.

Ankh : Optimized Protein Language Model Unlocks General-Purpose Modelling

Protein-specific tuning of masking, depth and positional embeddings, rather than more parameters, yields an encoder-decoder model that tops structure and function benchmarks at a fraction of the compute.

DeepTMHMM predicts alpha and beta transmembrane proteins using deep neural networks

Combining an ESM-1b encoder with a conditional random field decoder, DeepTMHMM classifies alpha-helical and beta-barrel transmembrane proteins and their topology, outperforming most compared tools, including on signal-peptide cleavage sites.

ProteinBERT: A universal deep-learning model of protein sequence and function

A ~16M-parameter protein language model, pretrained on ~106M UniRef90 proteins by denoising sequences and Gene Ontology annotations, rivals larger TAPE models using linear-cost global attention.

Language models enable zero-shot prediction of the effects of mutations on protein function

A single pre-trained protein language model, ESM-1v, scores mutations zero-shot about as well as family-specific DeepSequence across 41 deep mutational scans, with no experimental labels or retraining.

BERTology Meets Biology: Interpreting Attention in Protein Language Models

One attention head of a protein BERT model tracks 3D contacts (28% vs 1.3% background); other heads target binding sites, and deeper layers attend to higher-level properties.

Transforming the Language of Life: Transformer Neural Networks for Protein Prediction Tasks

A RoBERTa-style Transformer, pre-trained on 450K Swiss-Prot sequences in 18 hours on 4 V100 GPUs, rivals DeepFam at family classification and beats PIPR at human protein-protein interaction prediction.

Evaluating Protein Transfer Learning with TAPE

Across five protein tasks with evolution-aware splits, self-supervised pretraining helped nearly every model on every task, yet alignment-based features still beat learned embeddings on secondary structure and contact prediction.

Deep Recurrent Neural Network for Protein Function Prediction from Sequence

Bidirectional LSTMs read raw amino-acid sequence to call protein function without alignment; ten unannotated iron-sequestration candidates flagged in UniRef100 lowered free iron when expressed in E. coli.