Deep learning based retrieval system for gigapixel histopathology cases and open access literature

Abstract

Clinical practice is getting increasingly stressful for pathologists due to increasing complexity and time constraints. Histopathology is slowly shifting to digital pathology, thus creating opportunities to allow pathologists to improve reading quality or save time using Artificial Intelligence (AI)-based applications. We aim to enhance the practice of pathologists through a retrieval system that allows them to simplify their workflow, limit the need for second opinions, while also learning in the process. In this work, an innovative retrieval system for digital pathology is integrated within a Whole Slide Image (WSI) viewer, allowing to define regions of interest in images as queries for finding visually similar areas using deep representations. The back-end similarity computation algorithms are based on a multimodal approach, allowing to exploit both text information and content-based image features. Shallow and deep representations of the images were evaluated, the later showed a better overall retrieval performance in a set of 112 whole slide images from biopsies. The system was also tested by pathologists, highlighting its capabilities and suggesting possible ways to improve it and make it more usable in clinical practice. The retrieval system developed can enhance the practice of pathologists by enabling them to use their experience and knowledge to properly control artificial intelligence tools for navigating repositories of images for decision support purposes.