From Dataset Curation to Unified Evaluation: Revisiting Structure Prediction Benchmarks with PXMeter

Abstract

Recent advances in deep learning have significantly improved the accuracy of structure prediction for biomolecular complexes; however, robust evaluation of these models remains a major challenge. We introduce PXMeter, an open-source toolkit that support consistent and reproducible evaluation of diverse predictive models across a broad spectrum of biological complex structures. PXMeter provides a unified and reproducible benchmarking framework, offering valuable insights to support the ongoing improvement of structure prediction methods. We also present a high-quality benchmark dataset curated from recently deposited structures in the Protein Data Bank (PDB). These entries are manually reviewed to exclude non-biological interactions, ensuring reliable evaluation. Using these resources, we conducted a comprehensive benchmark of several structure prediction models, namely Chai-1, Boltz-1, and Protenix. Our benchmarking results demonstrate the advancements achieved by deep learning models, while also identifying ongoing challenges--especially in modeling protein-protein and protein-RNA interactions. Project Pagehttps://github.com/bytedance/PXMeter