ProtParts, an automated web server for clustering and partitioning protein datasets

Abstract

Data leakage originating from protein sequence similarity shared among train and test sets can result in model overfitting and overestimation of model performance and utility. However, leakage is often subtle and might be difficult to eliminate. Available clustering tools often do not provide completely independent partitions, and in addition it is difficult to assess the statistical significance of those differences. In this study, we developed a clustering and partitioning tool, ProtParts, utilizing the E-value of BLAST to compute pairwise similarities between each pair of proteins and using a graph algorithm to generate clusters of similar sequences. This exhaustive clustering ensures the most independent partitions, giving a metric of statistical significance and, thereby enhancing the model generalization. A series of comparative analyses indicated that ProtParts clusters have higher silhouette coefficient and adjusted mutual information than other algorithms using k-mers or sequence percentage identity. Re-training three distinct predictive models revealed how sub-optimal data clustering and partitioning leads to overfitting and inflated performance during cross-validation. In contrast, training on ProtParts partitions demonstrated a more robust and improved model performance on predicting independent data. Based on these results, we deployed the user-friendly web server ProtParts (https://services.healthtech.dtu.dk/services/ProtParts-1.0) for protein partitioning prior to machine learning applications. GRAPHICAL ABSTRACT O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=79 SRC="FIGDIR/small/603234v1_ufig1.gif" ALT="Figure 1"> View larger version (22K): org.highwire.dtl.DTLVardef@c0df9forg.highwire.dtl.DTLVardef@994c6borg.highwire.dtl.DTLVardef@68147eorg.highwire.dtl.DTLVardef@1198eab_HPS_FORMAT_FIGEXP M_FIG C_FIG