Unlocking hidden flaws in PPV and ACC: A step towards more reliable identification of protein complex

Abstract

As classic evaluation indexes, clustering-wise predictive positive value (PPV) and accuracy (ACC) have been widely used for the detection of protein complexes ([1]). However, we identified a critical error in their calculation, which can lead to inaccurate evaluation results under most conditions. Here, we elaborate on the problem of the original indexes and propose revised indexes PPVM (PPV Modified) and ACCM, which correct the identified error. Experiments demonstrate that revised indexes achieve higher reliability. Based on the new indexes, we reevaluated three state-of-the-art computational methods for protein complex detection on five benchmarks to provide a revised baseline to facilitate comparison of performance for algorithms developed later. The code and data involved in the experimental section of this paper can be found in https://github.com/hyx-1/PPV_M-and-ACC_M.