Unsupervised Neural Network-Based Image Stitching Method for Bladder Endoscopy

Bladder endoscopy enables the observation of intravesical lesion characteristics, making it an essential tool in urology. Image stitching techniques are commonly employed to expand the field of view of bladder endoscopy. Traditional image stitching methods rely on feature matching. In recent years, deep-learning techniques have garnered significant attention in the field of computer vision. However, the commonly employed supervised learning approaches often require a substantial amount of labeled data, which can be challenging to acquire, especially in the context of medical data. Both feature-based and unreliable supervised methods for cystoscopy image stitching are limited by their quality and the robustness of image stitching. This study proposes an unsupervised neural network-based image stitching method for bladder endoscopy that comprises two modules: an unsupervised alignment network and an unsupervised fusion network. In the unsupervised alignment network, we employed feature convolution, regression networks, and linear transformations to align images. In the unsupervised fusion network, we achieved image fusion from features to pixel by simultaneously eliminating artifacts and enhancing the resolution. Experiments demonstrated our methods consistent stitching success rate of 98.11% and robust image stitching accuracy at various resolutions. Our method eliminates sutures and flocculent debris from cystoscopy images, presenting good image smoothness while preserving rich textural features. Moreover, our method could successfully stitch challenging scenes such as dim and blurry scenes. Our application of unsupervised deep learning methods in the field of cystoscopy image stitching was successfully validated, laying the foundation for real-time panoramic stitching of bladder endoscopic video images. This advancement provides opportunities for the future development of computer-vision-assisted diagnostic systems for bladder cavities.