Quantitative characterization of cell niches in spatial atlases

Abstract

Spatial omics allow us to identify and analyze communities of cells coordinating specific functions within a tissue. While these communities, defined as cell niches, are fundamentally shaped by interactions between spatially neighboring cells, we lack computational frameworks that can leverage spatial omics data to quantitatively characterize niches based on cell interaction events. To address this, we introduce NicheCompass, a graph deep learning method designed based on the principles of cellular communication. NicheCompass not only identifies cell niches, but also learns and informs about the signaling events shaping the identity of these niches. Unlike existing methods, it uniquely characterizes niches by quantifying their activity of spatial gene programs which represent diverse mechanisms of cell-cell communication and transcriptional regulation, thereby uncovering the underlying cellular processes constituting each niche. We showcase a comprehensive workflow encompassing data integration, niche identification, and functional interpretation, and demonstrate that, with its biologically informed design, NicheCompass outperforms existing methods. NicheCompass is broadly applicable to spatial transcriptomics data, which we illustrate by mapping the architecture of diverse tissues during mouse embryonic development, and delineating basal (KRT14) and luminal (KRT8) tumor niches in human breast cancer. We further introduce fine-tuning-based spatial reference mapping, revealing an SPP1+ macrophage-dominated tumor niche in non-small cell lung cancer patients. Additionally, we extend NicheCompass to multimodal spatial profiling of gene expression and chromatin accessibility, identifying and characterizing distinct white matter niches in the mouse brain. Finally, we apply NicheCompass to a whole mouse brain spatial atlas with 8.4 million cells demonstrating its scalability and ability to build foundational, interpretable spatial representations for entire organs. Overall, NicheCompass provides a novel approach to the challenge of identifying and analyzing niches, and suggests a more rigorous niche definition grounded in the quantitative characterization of underlying cellular processes.