Fold or flop: quality assessment of AlphaFold predictions on whole proteomes

MotivationReliability of AlphaFold2 predictions is mainly assessed using the predicted Local Distance Difference Test (pLDDT). For model organisms, 30-40% of residues fall into the low-confidence pLDDT range. Moreover, pLDDT sometimes fails to flag physically implausible structures. This raises two questions: can more robust reliability indicators be identified, and do unreliable predictions share common structural or biophysical features? ResultsWe characterize protein structures through histograms of per-residue neighbor counts, and use the Wasserstein principal component analysis to define the arity map, and lightweight and informative 2D embedding of proteins in a dataset. Using AlphaFold-DB, we show that the arity map reveals three structurally and biophysically distinct populations (well-folded proteins, intrinsically disordered proteins, and physically implausible predictions). We also use our packing based encoding at the residue level to define abstraqt (Arity-Based STRuctural Arrangement Quality assessmenT), a per-residue scoring function complementing the pLDDT, assigning low scores to hallucinated helices and distorted beta strands while correctly scoring native like predictions. AvailabilityThe code to compute arity maps is available within the Structural Bioinformatics Library. See: AlphaFold analysis, and also Documentation, Applications, Installation guide. The code and data for rerunning analyses are made available from AlphaFold analysis, and Data.