scHDeepInsight: A Hierarchical Deep Learning Framework for Precise Immune Cell Annotation in Single-Cell RNA-seq Data

Abstract

Immune cell classification from single-cell RNA sequencing (scRNA-seq) presents significant challenges due to complex hierarchical relationships among cell types. We introduce scHDeepInsight, a deep learning framework that extends our previous scDeepInsight model by integrating a biologically-informed classification architecture with an adaptive hierarchical focal loss. The framework leverages our established method of transforming gene expression data into two-dimensional structured images for use with convolutional neural networks by effectively capturing both global and fine-grained transcriptomic features, overcoming the limitations of flat classification approaches that ignore hierarchical relationships between cell types. scHDeepInsight dynamically adjusts loss contributions to balance performance across the hierarchy levels. It also employs STACAS batch correction, robust random masking, and SHAP-based interpretability to enhance prediction accuracy and biological insight. Comprehensive benchmarking across seven diverse tissue datasets shows scHDeepInsight achieves an average accuracy of 93.2%, representing a 5.1 percentage point improvement over current state-of-the-art methods. The model successfully distinguishes 50 distinct immune cell subtypes with high accuracy, demonstrating proficiency for identifying rare and closely related cell subtypes. These advantages make scHDeepInsight a robust tool for high-resolution immune cell subtype characterization, well suited for detailed immune profiling in immunological studies.