Do protein representations encode variant function? Diagnosing residue-level protein attributions via embedding-space gradients

A common workflow in protein engineering involves building fitness predictors on protein language models (pLMs) to estimate a variant's fitness and interpret which residues drive it. Attribution methods such as integrated gradients address this second goal by assigning residues contributions to the variant's predicted fitness. However, without ground truth for residue importance, it is unclear whether these attributions reflect what the predictor learned about function. Here, we introduce IGLOO, a framework that scores residue-level attributions against synthetic fitness landscapes with known per-residue importance. Across experimental axes, high predictive accuracy did not guarantee faithful attributions. We identify model interventions that more than double attribution skill scores and raise TOPSIS by 29%, with similar patterns holding in real -amylase and {beta}-lactamase datasets. IGLOO provides a protocol for probing what pLMs learn and a guide for choosing defaults when using residue attributions to pick sequence edits and build more trustworthy protein models.