Group Heteroscedasticity - A Silent Saboteur of Power and False Discovery in RNA-Seq Differential Expression

Abstract

Despite the availability of several high-profile, state-of-the-art methods, analyzing bulk RNA-Seq data continues to face significant challenges. Evidence from recent studies has highlighted that popular differential expression (DE) tools, such as edgeR and DESeq2, are susceptible to an alarmingly high false discovery rate (FDR). These studies suggest that the FDR inflation observed in these models could be attributed to issues such as violations of parametric assumptions or an inability to effectively handle outliers in the data. Here, we argue that group heteroscedasticity can also contribute to this elevated FDR, a phenomenon largely overlooked by the research community. We introduce a novel statistical model, Robseq, designed for effective per-feature modeling in differential analysis, particularly when the assumption of group homoscedasticity is unmet. Robseq utilizes well-established statistical machinery from the robust statistics literature, including M-estimators to robustly estimate gene expression level changes and Huber-Cameron variance estimators to calculate robust standard errors in heteroscedastic settings. Additionally, it incorporates a degrees of freedom adjustment for the Welch t-statistic, based on Bell-McCaffreys recommendation, for inferential purposes, effectively addressing the problem of FDR inflation in RNA-Seq differential expression. Through detailed simulations and comprehensive benchmarking, we show that Robseq successfully maintains the false discovery and type-I error rates at nominal levels while retaining high statistical power compared to well-known DE methods. Analysis of population-level RNA-Seq data further demonstrates that Robseq is capable of identifying biologically significant signals and pathways implicated in complex human diseases that otherwise cannot be revealed by published methods. The implementation of Robseq is publicly available as an R package at https://github.com/schatterjee30/Robseq.