MutQA: A Cross-Validated Q&A Dataset for Genetic Mutations

Abstract

We introduce MutQA, a dataset of 330,387 citation-grounded question-answer (QA) pairs linking genetic mutations to their functional consequences as described in the biomedical literature. Each record pairs a protein-coding mutation with a natural-language question and an evidence-backed answer anchored to sentences in one of 27,442 open-access PubMed Central articles, spanning 6,620 genes and 125,593 gene-variant pairs. The dataset is constructed using CrossValQA, a framework that enforces information isolation between independent LLMs: one model generates QA pairs from an article, a second model answers the same question without access to the first model's response, and a third model judges whether each model's rationale supports the other's answer, so that retained QA pairs are reproducible from the source text rather than hallucinated. Of 401,406 audited QA pairs, 82.7% pass both directions of this cross-validation; those with a recoverable reference protein sequence form the released benchmark. In a 999-pair stratified audit, two expert annotators agree exactly on 86.4% of cross-validated QA pairs (95% CI 83.7-89.0; n = 638), a tier in which about 90% of ratings are "correct". Individual LLM judges agree with individual annotators at a comparable 87.0% on this tier but far less on QA pairs that fail cross-validation. Fine-tuning MutaPLM on MutQA nearly doubles its mean LLM-judge score for question-conditioned mutation-effect answering (1.45 to 2.82 on a 0-5 scale) and improves mutation-recovery accuracy from effect descriptions from 19.4% to 32.1%. Providing variant-matched MutQA retrieval context, without fine-tuning, improves strict answer accuracy across four LLM families by 20.0-28.6 percentage points, demonstrating the framework's ability to improve the reliability of LLM answers. We also apply CrossValQA to proteomics, providing a proof of concept for its use beyond mutation-specific QA. We release the dataset, construction framework, and evaluation code.