Interpreting prediction intervals and distributions for decoding biological generality in meta-analyses

Abstract

Despite the importance of identifying predictable regularities for knowledge transfer across contexts, the generality of ecological and evolutionary findings is yet to be systematically quantified. We present the first large-scale evaluation of generality using new metrics. By focusing on biologically relevant study levels, we show that generalization is not uncommon. Overall, 20% of meta-analyses will produce a non-zero effect 95% of the time in future replication studies with a 70% probability of observing meaningful effects in study-level contexts. We argue that the misconception that generalization is exceedingly rare is due to conflating within-study and between-study variances in ecological and evolutionary meta-analyses, which results from focusing too much on total heterogeneity (the sum of within-study and between-study variances). We encourage using our proposed approach to elucidate general patterns underpinning ecological and evolutionary phenomena.