Graph-theoretic comparisons of structural covariance networks: quantifying the false discovery rate

Structural covariance networks (SCNs) represent spatial patterns of covariation in brain morphology, often as a network of connections between nodes representing correlations in grey matter volume or cortical thickness measured by magnetic resonance imaging (MRI). SCNs have been suggested to reveal differences in functional organisation that are reflected in coordinated alterations to brain structure, and often these differences are sought in graph-theoretic measures such as the degree of clustering, segregation into distinct modules, or the characteristic path length between nodes. A common practice is to calculate SCNs for groups of interest, and use permutation testing to determine if they are significantly different for the measure of interest. However, the statistical validity of group comparisons using SCN-derived graph measures remains poorly understood. Here, we systematically evaluate the reliability of SCN estimation and downstream graph-theoretic analyses using structural MRI data from the Human Connectome Project (n = 1,096). We use simulations to show the effects of sample size and atlas dimensionality on SCN reliability. Using bootstrapping to characterise the distribution of SCN graph measures, we establish that small sample sizes systematically bias graph-theoretic measures including clustering, characteristic path length and modularity. Finally, we use simulations based on extrema from the bootstrapping distribution to characterise the statistical power and false discovery rate (FDR) for graph-theoretic between-group comparisons of SCNs, showing that at small sample sizes (n [≤] 30) permutation testing is no better than chance. These findings suggest that many significant SCN group differences, particularly those using small samples and high-dimensional parcellations, may reflect sampling noise rather than true biological differences. We recommend that future SCN studies use larger samples, coarser parcellations, and explicitly evaluate reliability before interpreting group differences.