MyoNet and AlveoliNet: A hybrid 3D neural-net pipeline for the instance segmentation and scoring of epithelial cells in intact mammary alveoli

Abstract

Recent advances in tissue clearing protocols such as DISCO, CUBIC, Clarity, FUnGI, and PEGASOS have revolutionized our ability to label and image intact 3-dimensional (3D) biological structures using fluorescence microscopy. The lactating mammary gland particularly benefits from clearing due to its high degree of tissue opacity. Cleared mammary gland images are strikingly beautiful and complicated but are difficult to fully interpret without developing a series of quantitative techniques and assays to analyze and compare them. These approaches will ultimately be as varied as the biology each scientist wishes to study. Here, we present one strategy based on a modular, hybrid, deep-learning approach and classical image processing that can segment and measure alveoli, cell nuclei and myoepithelial cells in intact mammary tissue. We have developed two original, three-dimensional (3D) U-shaped encoder-decoder networks (U-Nets), AlveoliNet and MyoNet, and combined these with CellPose3 nuclear instance segmentation and SlideBook/SlideBook Synergy binary mask operations. This approach can be used to easily score 100,000s of cells in intact tissue and differentiated glands at different developmental stages, genetic backgrounds, or treatments. We demonstrate the utility of this approach for quantifying the change in proportion of myoepithelial cells over the pregnancy-lactation transition, driven by endoreplication in the gland postpartum. We present a complete methods pipeline for other laboratories to utilize our approach in their own studies using standard desktop computers.