AI and ophthalmology

March 27, 2018 Dr Sheng Chiong Hong

From driverless cars to robot surgery, it’s clear this is the dawn of the age of machines, and medicine is becoming a primary focus for this artificial intelligence (AI) revolution.

In 2011, IBM (International Business Machines Corp.) introduced IBM Watson Health, to complement its first artificial question-answering supercomputer Watson, combining AI, or machine learning, and sophisticated analytical software. Watson Health’s processing abilities encompass a comprehensive package of cognitive healthcare solutions utilising big health data, machine learning and cloud analytics to provide what’s now commonly called “precision medicine” - the customisation of healthcare, with medical decisions, treatments, practices and/or products tailored to an individual patient.

Since then the number of companies and researchers globally who participate in healthcare machine learning has soared tremendously. But, despite the rapid development and positive publicity, this new smart health tech is still being met with a degree of criticism and scepticism among healthcare practitioners as there is very little good quality, evidence-based results data available. This is partly due to the lack of peer-reviewed published evidence and deep learning (machine creators’ attempts to mimic the thinking part of our brains to produce ‘real’ AI) being labelled as “black box” systems, ie. either they’re too complicated to understand or consisting of proprietary algorithms manufacturers refuse to share or explain.

AI and diabetic retinopathy detection

In 2016, Gulshan et al published a landmark paper in the field of clinical artificial intelligence¹. The study, which was funded by Alphabet (Google's parent company), involved the development of a deep learning algorithm for the detection of diabetic retinopathy (DR) in retinal fundus photographs. It used a clinical data set totalling 128,175 retinal images, which were graded three to seven times for diabetic retinopathy, diabetic macular oedema and image quality by a panel of 54 ophthalmologists and senior ophthalmology residents.

The research team used Inception-v3 architecture as their deep learning neural network. This was “trained” to make predictions as to whether the DR in the image was mild, moderate, severe or proliferative DR. The resultant algorithm was then validated with two separate data sets (EyePACS-1 and Messidor-2). Using the first operating cut point with high specificity, the EyePACS-1 yielded a sensitivity of 90.3% and specificity of 98.1%. For Messidor-2, the sensitivity was 87% and specificity 98.5%. Using a second operating point with high sensitivity in the development set for EyePACS-1, the sensitivity was 97.5% and specificity was 93.4% and for Messidor-2, the sensitivity was 96.1% and specificity was 93.9%. Thus, the authors concluded the algorithm based on deep machine learning had high sensitivity and specificity for detecting referable DR.

The positive findings from this Google-funded study have led to a major partnership being formed between Verily (the life sciences division of Google) and Nikon. The plan is to combine Nikon’s Optos wide-field retinal imaging technology with Verily’s machine-learning technology to develop the next generation of affordable retinal cameras incorporating diagnostic AI, and thus to be able to diagnose DR and other retinal diseases far earlier than is currently possible.

Google’s artificial intelligence is far from magic or new, however. The concept of machine learning, and more specifically deep learning, has been around for over 30 years. But it’s only in the last decade that computing power has actually been up to the task, following the introduction of powerful graphics processing units (GPUs). Google is not alone in its focus on AI as the way of the future, with other companies such as Microsoft, Nvidia, IBM and Amazon introducing GPUs into the cloud to enable better machine learning.

In New Zealand, Dr Nishan Ramachandran and his team evaluated a deep learning diabetic screening system, Visiona Intelligent, with diabetic retinal photos acquired from the Otago diabetic screening database (485 photos)². The Visiona Intelligent system was developed by Visiona Medtech in Hong Kong and was “trained” with a dataset of more than 100,000 fundus photographs, with more than 30% graded as severe enough to be referred to an ophthalmologist. The Visiona system was also used to evaluate the Messidor database. Visiona’s ability to detect referable DR was (area under curve) 0.901 for the Otago database and 0.98 for the Messidor database, so the results were comparable to the system developed by Google. However, the system is not capable of more specific tasks such as detecting other common ocular pathologies such as vessel-occlusive diseases, macular degeneration or glaucoma.

The use of convolutional neural networks (CNN) in machine learning has been grabbing the headlines globally. The idea of CNNs come from experiments on cats’ visual cortexes, where it was discovered that lower level neurons in the cortex detect lower level features, while higher level neurons in the cortex find higher level features. This is how humans also classify images, by learning different features through example.

At present, most machine learning is still supervised learning, where we give a machine labelled data, but it’s expected a time will come when unsupervised learning will supersede supervised learning, allowing machines to teach themselves through experience.

AI and other eye diseases

Fundal photograph auto-diagnosis is not the only domain where deep learning can tackle areas efficiently. Several investigators have published promising results with AI in other areas such as age-related macular degeneration AMD) and glaucoma.

Burlina et al published an abstract on the use of deep learning for recognition of age-related macular degeneration (AMD)³ and presented it at the 13^th International Symposium on Biomedical Imaging in Prague in 2016. In this study the AI neural network was tested using over 5600 images from the US National Institutes of Health Age-Related Eye Disease Study (AREDS) dataset with preliminary results for accuracy ranging between 92% and 95%.

Another deep learning and AMD study by the ophthalmology department at the University of Washington’s School of Medicine extracted 2.6 million OCT images from American electronic medical records and used a total of 80,839 images (41,074 from AMD and 39,765 from normal patients) for training⁴. Again, investigators reported a high accuracy of 93.45%, with a sensitivity of 83.82% and a specificity of 96.40%.

An interesting and eye-catching system called ALADDIN (automatic feature learning for glaucoma detection based on deep learning) was developed by a team from Singapore⁵. Led by Professor Tien Yin Wong, the system was designed with a deep learning framework designed to generate a multi-layer neural network of both linear and non-linear transformations of the data to yield more abstract and useful representations. The system was tested extensively on the ORIGA and SCES datasets with results showing area under curve (AUC) of the receiver operating characteristic curve in glaucoma detection of 0.838 and 0.898 in the two databases, leading the authors to conclude that the system performs better than present state-of-the-art algorithms and can be used for glaucoma detection and diagnosis.

Asaoka et al published a paper on the use of deep learning for detecting pre-perimetric glaucoma with standard automated perimetry⁶. A total of 171 pre-perimetric glaucoma visual fields were obtained and tested against 108 healthy visual fields. The authors compared the results of deep feed-forward neural network (FNN) with older machine learning classifiers, such as random forests (RF), gradient boosting, support vector machine (SVM) and neural network (NN). The authors reported a 92.5% of AUC value for deep FNN and concluded that the deep learning training technique can distinguished pre-perimetric glaucoma visual fields from health visual fields with very high accuracy.

Another glaucoma detection system based on a deep learning framework called Glaucoma-Deep was published by Qairsar Abbas⁷. A total of 1200 retinal images were selected from both public and private datasets (600 normal and 600 with glaucoma). The author reported a high accuracy of 99% with sensitivity of 84.5% and specificity of 98.01%. So far this is the only reported system with the greatest performance in detecting glaucoma based purely on retinal images.

Potential applications of deep learning

There are no limits to the types of conditions and diagnoses that deep learning techniques can be applied to. The same technology can be applied to a variety of conditions such as retinopathy of prematurity, retinitis pigmentosa, keratoconus (corneal topography), vessel-occlusive diseases and others. It can also be added to existing ophthalmic systems, such as retinal cameras, corneal topographers and OCT scanners.

The use of deep learning is also not limited to image classification. It started off with numerical analysis and is extremely useful in the analysis of big data. From a numerical standpoint, optimising biometry for cataract surgery is well within its capability. Deep learning also has the potential to outperform all existing methods of determining optimal intra-ocular lens power and can be applied to other predictive tasks, such as estimating the individual risk of cataract surgery, progression of disease or prognosis.

Another breakthrough in the use of deep convolutional neural networks for image super-resolution is SRGAN, a generative adversarial network (GAN) for image super-resolution (SR)⁸. A highly challenging task of estimating a high-resolution image from its low-resolution counterpart is referred to as super-resolution. For example, researchers may have been given a dataset of clinical images with lower resolution due to constrains such as storage size and internet bandwidth. With SRGAN, these images can be digitally optimised making them hard to distinguish from the original RAW files. SRGAN may also have a role in telemedicine, where clinical images are usually compressed for digital transfer to optimise speed.

For numerical and big data analysis, IBM’s Watson offers a range of solutions on the cloud including a capability for image classification, while Neural Designer (Artificial Intelligence Techniques Ltd, Spain), a desktop application, offers an advanced analytics platform for dealing with big data without programming or building block diagrams, though has no image classification capability.

Public retinal image databases

There is no doubt that an important requirement for deep learning is high quality data and a lot of it. The accuracy of a deep neural network depends on three major factors: the quality of the data, the quantity of the data, and the types of training implemented.

Health technology and AI is becoming the new gold rush, and good quality data can be more valuable than gold. Fortunately, for beginners who would like to start experimenting with deep learning, there is now a large number of high quality datasets available publicly (see Table 1).

Name	Description	Available from (website)
ORIGA	650 images acquired through the Singapore Malay Eye Study (SiMES), includes 168 glaucomatous and 482 non-glaucoma images	http://imed.nimte.ac.cn/Origa-650.html
High-resoltuon fundus (HRF) image database	15 images of healthy patients, 15 images of DR patients and 15 images of glaucomatous patients	https://www5.cs.fau.de/research/data/fundus-images/
DRIVE	40 images: 33 normal and seven with signs of mild/early diabetic retinopathy	https://www.isi.uu.nl/Research/Databases/DRIVE/download.php
Messidor	1200 retinal images graded for normal, DR and risk of diabetic macular oedema	http://www.adcis.net/en/Download-Third-Party/Messidor.html
Messidor-2	An extension of the Messidor database, containing 1748 retinal images	http://latim.univ-brest.fr/indexfce0.html
DIARETDB0	130 retinal images: 20 normal and 110 with signs of diabetic retinopathy	http://www.it.lut.fi/project/imageret/diaretdb0/
STARE	400 retinal images with a large variety of diagnoses: emboli, retinal vessel occlusive diseases, DR, hypertensive retinopathy and choroidal neovascularisation	http://cecas.clemson.edu/~ahoover/stare/
DIARETDB1	89 retinal images: 84 with at least mild non-proliferative signs of DR and five normal	http://www.it.lut.fi/project/imageret/diaretdb1/
EyePacs DR	35,126 retinal images from diabetic screening programmes	https://www.kaggle.com/c/diabetic-retinopathy-detection/data

Table 1. Publicly available retinal image databases

New Zealand-Based Deep Learning Platform – MedicMind

 

MedicMind is a web-based platform for image classification using Tensorflow deep learning framework

 

It can be difficult for clinical researchers to make use of deep learning in their research since conventional deep learning frameworks such as Tensorflow, Caffe, Theano and Torch require advanced knowledge of programming languages like C++ and Python. This would usually mean the services of a software engineer, or programmer with experience in one of the above deep learning frameworks, is needed who can be expensive and hard to find. MedicMind, however, offers a solution for clinical researchers with no prior knowledge of programming or coding. It is a free-to-use web-based system using Google’s deep mind framework Tensorflow with the best practice convolutional network, Inception V-3. The user interface has been designed to be as intuitive as possible with a simple drag and drop approach for importing image datasets. Researchers can easily develop state-of-the-art, deep neural networks with the platform with rich statistical analysis such as specificity, sensitivity and ROC curves. Statistics can also be easily exported for use in research and the deep neural network can be shared and exported as an external web-based app for external use, so a DR screening system can be trained and developed for resource-limited regions with minimal cost and time. MedicMind is available from http://www.medicmind.tech/

References

1. Gulshan v, Peng L, Coram M, et al. Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA. 2016; 316 (22): 2402-2410. Available from https://jamanetwork.com/journals/jama/fullarticle/2588763
2. Ramachandran N, Hong SC, Sime MJ, Wilson G. Diabetic Retinopathy Grading Using Deep Neural Network. Clinical and Experimental Ophthalmology. 2017. doi:10.1111/ceo 13056. Available from http://onlinelibrary.wiley.com/doi/10.1111/ceo.13056/abstract
3. Burlina P, Freund D E, Joshi N, et al. Detection of age-related macular degeneration via deep learning. 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI). 2016. Access available from http://ieeexplore.ieee.org/abstract/document/7493240/?reload=true
4. Lee CS, Baughman DM, Lee AY. Deep Learning Is Effective for Classifying Normal versus Age-related Macular Degeneration OCT Images. Ophthalmology Retina. 2017; 1(4) : 322 - 327. Available from http://www.ophthalmologyretina.org/article/S2468-6530(16)30174-9/pdf
5. Chen X, Xu Y, Yan S et al. Automatic Feature Learning for Glaucoma Detection Based on Deep Learning. International Conference on Medical Image Computing and Computer Assisted Intervention. 2015. pp 669 - 677. Available from https://link.springer.com/chapter/10.1007/978-3-319-24574-4_80
6. Asaoka R, Murata H, Araie M. Detecting Preperimetric Glaucoma with Standard Automatic Perimetry Using a Deep Learning Classifier. Ophthalmology. 2016. 123(9): 1974-1980. Available from https://www.ncbi.nlm.nih.gov/pubmed/27395766
7. Abbas Q. Glaucoma-Deep : Detection of Glaucoma Eye Disease on Retinal Fundus Images using Deep Learning. International Journal of Advanced Computer Science and Applications. 2017; 8(6). Available from https://pdfs.semanticscholar.org/fbc2/7038b9c111dad2851397047c6230ece79c23.pdf
8. Ledig C, Theis L, Huszar F et al. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. ARXIV. arXiv:1609.04802 Available from https://arxiv.org/abs/1609.04802

*Dr Sheng Chiong Hong (better known as Hong) is an ophthalmology registrar at Dunedin Hospital and co-founder and CEO of social eye care enterprise oDocs Eye Care and medical technology firms Vise Medic and Sober-Eye AV.