Scalable multi-group nonnegative spatial factorization for spatial genomics data with cell-type heterogeneity

Spatial transcriptomics (ST) technologies enable the study of gene expression within the spatial context of tissues, providing insights into tissue structure, cellular interactions, and disease progression. However, existing dimension reduction methods often overlook spatial information or struggle to distinguish spatial gene patterns from those driven by cell-type differences, limiting biological interpretability by convolving differences in gene expression patterns with differences in cell-type proportions. To address these challenges, we introduce the scalable multi-group nonnegative spatial factorization (smNSF), a computationally-tractable probabilistic framework that integrates spatial coordinates and cell-type labels into a unified matrix factorization model. By using multi-group Gaussian processes (MGGPs) as priors, our model captures complex spatial variation in a cell-type specific way while enforcing nonnegativity to enhance interpretability. We develop a variational inference framework for MGGPs that supports scalable optimization and improves the numerical stability of smNSF. Across seven spatial transcriptomics datasets spanning diverse technologies and tissues, smNSF recovers sparse, interpretable spatial factors and, through its cell-type conditional posteriors, organizes them into cell-type enriched, cell-type specific, and universal spatial programs that are not apparent from marginal factors alone. Given cell-type labels in ST data, smNSF enables cell-type aware spatial decompositions and supports cell-type conditional posteriors for in silico exploration of relationships between spatial patterns and cellular identity. Author summaryMost current analysis methods for spatial transcriptomics either ignore spatial structure or fail to separate gene-driven spatial patterns from cell-type driven differences. In this work, we develop a method that uses spatial coordinates and cell-type labels together to better uncover patterns of gene expression. Our approach, scalable multi-group nonnegative spatial factorization (smNSF), uses Gaussian processes to model spatial structure, which we extend to capture both spatial structure and cell type within a unified framework called multi-group Gaussian processes. Applying smNSF to spatial transcriptomics datasets from mouse brain and human tissues, we find that conditioning on different cell types reveals spatial patterns that are invisible in standard analyses: some cell types suppress a pattern, others sharpen it, and some reveal structure that only emerges after conditioning. This helps us understand how cell-type specific spatial programs contribute to tissue organization. To make this analysis tractable on the scale of modern spatial experiments, we also introduce a new computational approximation for Gaussian processes that is directly applicable in principle to other latent variable GP models. Together, these tools help disentangle biological sources of variation and support in silico exploration of how gene expression might change under different tissue or cell-type compositions.