Save GPU RAM Usage in Convolutional Layers to Load Huge Images

Abstract

Image recognition models have evolved tremendously. Despite the progress for general images, histopathological images are not easy targets. One of the reasons is that histopathological images can be 100000-200000px in height and width which are often too large for a deep neural network model to handle directly because the RAM of the GPU is limited. Mitigating the obstacle is expected to be a progress in the histopathological image analysis. In this study, we save the RAM consumption in a convolutional layer by allocating only the required data to GPU only when needed and by dividing the calculation into per channel. This RAM Saving Convolutional layer (RSConv) can load larger images than a normal convolutional layer. The code is available at https://github.com/tand826/RAMSavingConv2d.