Deep learning on reflectance confocal microscopy improves Raman spectral diagnosis of basal cell carcinoma

SignificanceRaman spectroscopy may be useful to assist Mohs micrographic surgery for skin cancer diagnosis; however, the specificity of Raman spectroscopy is limited by the high spectral similarity between tumors and normal tissues structures such as epidermis and hair follicles. Reflectance confocal microscopy (RCM) can provide imaging guidance with morphological and cytological details similar to histology. Combining Raman spectroscopy with deep-learning-aided RCM has the potential to improve the diagnostic accuracy of Raman without requiring additional input from the clinician. AimWe seek to improve the specificity of Raman for basal cell carcinoma (BCC) by integrating information from RCM images using an Artificial Neural Network. ApproachA Raman biophysical model was used in prior work to classify BCC tumors from surrounding normal tissue structures. 191 RCM images were collected from the same site as the Raman data and served as inputs to train two ResNet50 networks. The networks selected the hair structure images and epidermis images respectively within all the images corresponding to the positive predictions of the Raman Biophysical Model. ResultsDeep learning on RCM images removes 54% of false positive predictions from the Raman Biophysical Model result and keeps the sensitivity as 100%. The specificity was improved from 84.8% by using Raman spectra alone to 93.0% by integrating Raman spectra with RCM images ConclusionsCombining Raman spectroscopy with deep-learning-aided RCM imaging is a promising tool to guide tumor resection surgery.