ClustSIGNAL identifies cell types and subtypes using an adaptive smoothing approach for scalable spatial omics clustering

Abstract

Unsupervised clustering of spatial transcriptomics data is challenging due to sparsity and complex tissue structures. We introduce ClustSIGNAL spatial clustering method that employs neighbourhood complexity-based expression smoothing. ClustSIGNAL uses entropy to measure neighbourhood compositions and generate cell-specific weights that control the effective smoothing radius within neighbourhoods. Simulation analysis demonstrates its robustness to segmentation errors and sparsity. Benchmarking across diverse SRT datasets shows its ability to perform scalable, multi-sample clustering of atlas-level data. By stabilising expression in homogeneous regions and preserving distinct expression in heterogeneous regions, ClustSIGNAL avoids over-smoothing and identifies biologically meaningful spatially-informed cell states. ClustSIGNAL Bioconductor package: bioconductor.org/packages/clustSIGNAL.