Real-Time Multi-Position and Multi-ROI Tracking with LiLiTTool for Smart Light-Sheet Microscopy in Growing Samples

Abstract

Long-term live imaging of growing samples with light-sheet fluorescence microscopy provides unique insights into development, but morphogenesis often displaces features of interest outside the microscopes field of view (FOV), calling for automated methods to track these features and update the microscopes FOV in real time. Existing solutions, which typically rely on local or global intensity distributions, struggle to follow specific features robustly throughout morphogenesis, leading to truncated or incomplete datasets. Here, we present a light-sheet live tracking tool (LiLiTTool) that maintains user-defined regions of interest (ROI) within the FOV throughout extended imaging sessions. LiLiTTool uses Cotracker3, a state-of-the art deep learning-based motion predictor, augmented by sensor fusion with a trained object-detector. This enables robust compensation for drift, rotation, and deformation during morphogenesis, while meeting the timing constraints of live acquisition. We validated LiLiTool by integrating with the Viventis LS1 microscope, achieving sub-second processing and stable tracking of growing zebrafish embryos over many hours. LiLiTTool supports multi-ROI tracking in 3D, enabling simultaneous monitoring of multiple features within the same embryo and in multiple embryos during a single acquisition. LiLiTTool is modular and openly available on GitHub and as a napari plugin for post-acquisition tracking. By enabling precise, adaptive, and scalable real-time imaging, LiLiTTool advances smart microscopy approaches and provides the developmental biology community with a practical tool for capturing reliable spatio-temporal information in growing embryos or other morphogenetic systems.