BIG-TB: A benchmark for evaluating prediction and interpretability of sequence-based machine learning using Mycobacterium tuberculosis genomes

Foundation models aim to learn useful representations of biological sequences, but are hindered by a lack of benchmarks to jointly evaluate predictive performance and variant attribution on diverse tasks. To this end, we present the Benchmark for Interpretable prediction from Genomes of Tuberculosis, BIG-TB. We curate over 17,000 genomes with high-quality sequencing data and experimentally measured antibiotic resistance phenotypes, combined with a curated list of WHO-catalogue of resistance-associated variants, and provide these data in an ML-ready format. BIG-TB defines two tasks: (1) predictive performance of antibiotic resistance phenotypes, and (2) attribution of predictions to known resistance-conferring variants. Using our tasks, we evaluate DNA- and protein-based foundation models, showing that DNA-based foundation models do not yet outperform simple machine learning baselines (mean test AUC = 0.888 vs. 0.865 for best CNN variant vs. best Evo2 variant across drugs). In a lineage-informed evaluation across DNA and protein modalities, predictive signal for well-characterized drugs was largely preserved (mean AUC 0.846-0.963). Across both evaluation settings, models with the highest predictive performance did not necessarily achieve the strongest recovery of resistance variants, underscoring the value of evaluating both predictive accuracy and biological interpretability. Overall, BIG-TB provides a reusable resource for developing sequence representations that are both accurate and mechanistically meaningful.