Robust chromatin state annotation

BackgroundSegmentation and genome annotations (SAGA) methods such as ChromHMM and Segway are widely to annotate chromatin states in the genome. These algorithms take as input a collection of genomics datasets, partition the genome, and assign a label to each segment such that positions with the same label have similar patterns in the input data. SAGA methods output an human-interpretable summary of the genome by labeling every genomic position with its annotated activity such as Enhancer, Transcribed, etc. Chromatin state annotations are essential for many genomic tasks, including identifying active regulatory elements and interpreting disease-associated genetic variation. However, despite the widespread applications of SAGA methods, no principled approach exists to evaluate the statistical significance of SAGA state assignments. ResultsTowards the goal of producing robust chromatin state annotations, we performed a comprehensive evaluation of the reproducibility of SAGA methods. We show that SAGA annotations exhibit a large degree of disagreement, even when run with the same method on replicated data sets. This finding suggests that there is significant risk to using SAGA chromatin state annotations. To remedy this problem, we introduce SAGAconf, a method for assigning a measure of confidence (r-value) to SAGA annotations. This r-value is assigned to each genomic bin of a SAGA annotation and represents the probability that the label of this bin will be reproduced in a replicated experiment. This process is analogous to irreproducible discovery rate (IDR) analysis that is commonly used for ChIP-seq peak calling and related tasks. Thus SAGAconf allows a researcher to select only the reliable parts of a SAGA annotation for use in downstream analyses. SAGAconf r-values provide accurate confidence estimates of SAGA annotations, allowing researchers to filter out unreliable elements and remove doubt in those that stand up to this scrutiny.