Optical Phenotyping Using Label-Free Microscopy and Deep Learning

SignificanceTissue phenotyping plays a critical role in biomedical research and clinical applications by providing insight into the structural and functional characteristics of tissues that can characterize clinical behavior and identify therapeutic targets. However, conventional phenotyping techniques are destructive, time-intensive, and expensive, posing challenges for both efficiency and widespread use. AimTo develop an optical phenotyping approach in pancreatic cancer specimens using label-free multiphoton microscopy combined with spatial transcriptomics and deep learning. ApproachWe measure and co-register a dataset comprised of spatial transcriptomics, autofluorescence, and second harmonic generation microscopy. We then cluster tissue subregions into meaningful phenotypes using transcriptomic signatures. We then evaluate three different classification models to predict phenotype based on label-free imaging data, and assess generalizability and prediction accuracy. ResultOur deep-learning classification model achieves over 89% accuracy in classifying six tissue types using label-free microscopy images. The one-vs-rest AUC values for all classes approaches 1, confirming the robustness of our model. ConclusionWe demonstrate the feasibility of optical phenotyping in distinguishing the structural and functional characteristics of pancreatic cancer specimens. Integrating additional gene-expression data or complementary label-free imaging modalities, such as fluorescence lifetime imaging microscopy, holds the potential to further enhance its accuracy and expand its applications in clinical research and diagnostics. Statement of DiscoveryWe develop and demonstrate a method for optical phenotyping using deep learning to classify label-free microscopy images into tissue phenotypes defined by transcriptomic signatures.