A statistical framework for measuring reproducibility and replicability of high-throughput experiments from multiple sources

Abstract

SO_SCPLOWUMMARYC_SCPLOWReplication is essential to reliable and consistent scientific discovery in high-throughput experiments. Quantifying the replicability of scientific discoveries and identifying sources of irreproducibility have become important tasks for quality control and data integration. In this work we introduce a novel statistical model to measure the reproducibility and replicability of findings from replicate experiments in multi-source studies. Using a nested copula mixture model that characterizes the interdependence between replication experiments both across and within sources, our method quantifies reproducibility and replicability of each candidate simultaneously in a coherent framework. Through simulation studies, an ENCODE ChIP-seq dataset and a SEQC RNA-seq dataset, we demonstrate the effectiveness of our method in diagnosing the source of discordance and improving the reliability of scientific discoveries.