GCTransNet: 3D mitochondrial instance segmentation based on Global Context Vision Transformers

Abstract

Mitochondria are double membrane-bound organelles essential for generating energy in eukaryotic cells. Mitochondria can be readily visualized in 3D using Volume Electron Microscopy (vEM), and accurate image segmentation is vital for quantitative analysis of mitochondrial morphology and function. To address the challenge of segmenting small mitochondrial compartments in vEM images, we propose an automated mitochondrial segmentation method called GCTransNet. This method employs grayscale migration technology to preprocess images, effectively reducing intensity distribution differences across EM images. By utilizing 3D Global Context Vision Transformers (GC-ViT) combined with global context self-attention modules and local self-attention modules, GCTransNet precisely models long-range and short-range spatial interactions. The long-range interactions enable the model to capture the global structural relationships within the mitochondrial segmentation network, while the short-range interactions refine local details and boundaries. In our approach, the encoder of the 3D U-Net network, a classical multi-scale learning architecture that retains high-resolution features through skip connections and combines multi-scale features for precise segmentation, is replaced by a 3D GC-ViT. The GC-ViT leverages shifted window-based self-attention, capturing long-range dependencies and offering improved segmentation accuracy compared to traditional U-Net encoders. In the MitoEM mitochondrial segmentation challenge, GCTransNet achieved state-of-the-art results, demonstrating its superiority in automated mitochondrial segmentation. The code and its documentation are publicly available at https://github.com/GanLab123/GCTransNet.