Why do some predicted protein structures fold poorly? Benchmarking AlphaFold, ESMFold, and Boltz in maize

Abstract

Protein structure prediction tools have significantly reduced the time and cost to generate protein structures and accelerated protein discovery and design. However, plant proteins are underrepresented in sequence and structural datasets used to train these programs. To quantify the downstream impact of this deficiency, we benchmarked five structure-prediction programs (AlphaFold 2, AlphaFold 3, ESMFold, Boltz-1, and Boltz-2) across 417 well-characterized Zea mays genes. These "classical" genes represent a set of well-studied genes with known genetic and phenotypic effects. We generated structures for each gene using these programs and compared how sequence, structural, and evolutionary conservation impacted the structures confidence and geometric features. Proteins lacking conserved sequence and/or structural domains had on average 25% to 43% lower confidence scores than proteins having both domains. Proteome-wide phylostratigraphy revealed that species-specific proteins had substantially lower confidence scores than proteins conserved amongst angiosperms and Eukaryotes. Boltz-1 and ESMFold structures had the highest occurrence of structures with severe geometry issues, including overlapping atoms and unlikely bond angles. We also compared computational and experimental alignments of Arabidopsis, maize, rice, wheat, and soybean proteins from the Protein Data Bank and identified structures showing incongruences with experimental data. This study challenges the assertion that protein folding has been completely solved, and urges more investigation into benchmarking and standardized evaluation frameworks to improve model performance and assessment in agricultural crops.