Unsupervised Multi-scale Segmentation of Cellular Cryo-electron Tomograms with Stable Diffusion Foundation Model

Abstract

We introduce an unsupervised approach for segmenting multiscale subcellular objects in 3D volumetric cryo-electron tomography (cryo-ET) images, addressing key challenges such as large data volumes, low signal-to-noise ratios, and the heterogeneity of subcellular shapes and sizes. The method requires users to select a small number of slabs from a few representative tomograms in the dataset. It leverages features extracted from all layers of a Stable Diffusion foundation model, followed by a novel heuristic-based feature aggregation strategy. Segmentation masks are generated using adaptive thresholding, refined with CellPose to split composite regions, and then utilized as pseudo-ground truth for training deep learning models. We validated our pipeline on publicly available cryo-ET datasets of S. Pombe and C. Eleganscell sections, demonstrating performance that closely approximates expert human annotations. This fully automated, data-driven framework enables the mining of multi-scale subcellular patterns, paving the way for accelerated biological discoveries from large-scale cellular cryo-ET datasets.