A positive statistical benchmark to assess network agreement

Abstract

As the current best practice, an experimental network dataset is validated by showing significant overlap with a gold standard network. Significance is assessed by comparison to a negative benchmark, often a randomized version of the same gold standard. While such analysis can reliably indicate the presence of signal, it is illsuited to assess how much signal there is. As an alternative, here we introduce a positive statistical benchmark corresponding to the best-case scenario, capturing the maximum possible overlap between two networks. Such a positive benchmark can be efficiently generated in a maximum entropy framework and opens the way to assess if the observed overlap is significantly different from the best-case scenario. In combination with the negative benchmark, we provide a normalized overlap score (Normlap). As an application, we compare molecular and functional networks, resulting in an agreement network of human as well as yeast network datasets. Although the number of shared interactions between most networks within the same organism is relatively small, we find that it is typically close to the best-case scenario. This paradox is resolved by the underlying degree inconsistency, meaning that highly connected hubs in one network often have small degrees in another, limiting the potential overlap. Furthermore, we illustrate how Normlap improves the quality assessment of experimental networks, fostering the creation of future high-quality networks.