An Ensemble Learning and Slice Fusion Strategy for Three-Dimensional Nuclei Instance Segmentation

Automated microscopy image analysis is a fundamental step for digital pathology and computer aided diagnosis. Most existing deep learning methods typically require post-processing to achieve instance segmentation and are computationally expensive when directly used with 3D microscopy volumes. Supervised learning methods generally need large amounts of ground truth annotations for training whereas manually annotating ground truth masks is laborious especially for a 3D volume. To address these issues, we propose an ensemble learning and slice fusion strategy for 3D nuclei instance segmentation that we call Ensemble Mask R-CNN (EMR-CNN) which uses different object detectors to generate nuclei segmentation masks for each 2D slice of a volume and propose a 2D ensemble fusion and a 2D to 3D slice fusion to merge these 2D segmentation masks into a 3D segmentation mask. Our method does not need any ground truth annotations for training and can inference on any large size volumes. Our proposed method was tested on a variety of microscopy volumes collected from multiple regions of organ tissues. The execution time and robustness analyses show that our method is practical and effective.