Image-based consensus molecular subtype classification (imCMS) of colorectal cancer using deep learning

Abstract

Image analysis is a cost-effective tool to associate complex features of tissue organisation with molecular and outcome data. Here we predict consensus molecular subtypes (CMS) of colorectal cancer (CRC) from standard H&E sections using deep learning. Domain adversarial training of a neural classification network was performed using 1,553 tissue sections with comprehensive multi- omic data from three independent datasets. Image-based consensus molecular subtyping (imCMS) accurately classified CRC whole-slide images and preoperative biopsies, spatially resolved intratumoural heterogeneity and provided accurate secondary calls with higher discriminatory power than bioinformatic prediction. In all three cohorts imCMS established sensible classification in CMS unclassified samples, reproduced expected correlations with (epi)genomic alterations and effectively stratified patients into prognostic subgroups. Leveraging artificial intelligence for the development of novel biomarkers extracted from histological slides with molecular and biological interpretability has remarkable potential for clinical translation.