Towards implementation of AI in New Zealand national screening program: Cloud-based, Robust, and Bespoke

Abstract

Convolutional Neural Networks (CNN)s have become a prominent method of AI implementation in medical classification tasks. Grading Diabetic Retinopathy (DR) has been at the forefront of the development of AI for ophthalmology. However, major obstacles remain in the generalization of these CNNs onto real-world DR screening programs. We believe these difficulties are due to use of 1) small training datasets (<5,000 images), 2) private and curated repositories, 3) offline CNN implementation methods, while 4) relying on accuracy measured as area under the curve (AUC) as the sole measure of CNN performance.\n\nTo address these issues, the public EyePACS Kaggle Diabetic Retinopathy dataset was uploaded onto Microsoft Azure cloud platform. Two CNNs were trained as a \"Quality Assurance\", and a \"Classifier\". The \"Classifier\" CNN performance was then tested both on un-curated as well as the curated test set created by the \"Quality Assessment\" CNN. Finally, the sensitivity of the \"Classifier\" CNNs was boosted post-training using two post-training techniques.\n\nOur \"Classifier\" CNN proved to be robust, as its performance was similar on curated and uncurated sets. The implementation of cascading thresholds and max margin techniques led to significant improvements in the \"Classifier\" CNNs sensitivity, while also enhancing the specificity of other grades.