A scalable and unbiased discordance metric with H+

Abstract

A standard unsupervised analysis is to cluster observations into discrete groups using a dissimilarity measure, such as Euclidean distance. If there does not exist a ground-truth label for each observation necessary for external validity metrics, then internal validity metrics, such as the tightness or consistency of the cluster, are often used. However, the interpretation of these internal metrics can be problematic when using different dissimilarity measures as they have different magnitudes and ranges of values that they span. To address this problem, previous work introduced the scale-agnostic G+ discordance metric, however this internal metric is slow to calculate for large data. Furthermore, we show that G+ varies as a function of the proportion of observations in the predicted cluster labels (group balance), which is an undesirable property. To address this problem, we propose a modification of G+, referred to as H+, and demonstrate that H+ does not vary as a function of group balance using a simulation study and with public single-cell RNA-sequencing data. Finally, we provide scalable approaches to estimate H+, which are available in the fasthplus R package.