Мethods of Machine Learning in Ophthalmology: Review

Machine learning is applied in every field of human activity using digital data. In recent years, many papers have been published concerning artificial intelligence use in classification, regression and segmentation purposes in medicine and in ophthalmology, in particular. Artificial intelligence is a subsection of computer science and its principles, and concepts are often incomprehensible or used and interpreted by doctors incorrectly. Diagnostics of ophthalmology patients is associated with a significant amount of medical data that can be used for further software processing. By using of machine learning methods, it’s possible to find out, identify and count almost any pathological signs of diseases by analyzing medical images, clinical and laboratory data. Machine learning includes models and algorithms that mimic the architecture of biological neural networks. The greatest interest in the field is represented by artificial neural networks, in particular, networks based on deep learning due to the ability of the latter to work effectively with complex and multidimensional databases, coupled with the increasing availability of databases and performance of graphics processors. Artificial neural networks have the potential to be used in automated screening, determining the stage of diseases, predicting the therapeutic effect of treatment and the diseases outcome in the analysis of clinical data in patients with diabetic retinopathy, age-related macular degeneration, glaucoma, cataracts, ocular tumors and concomitant pathology. The main characteristics were the size of the training and validation datasets, accuracy, sensitivity, specificity, AUROC (Area Under Receiver Operating Characteristic Curve). A number of studies investigate the comparative characteristics of algorithms. Many of the articles presented in the review have shown the results in accuracy, sensitivity, specificity, AUROC, error values that exceed the corresponding indicators of an average ophthalmologist. Their introduction into routine clinical practice will increase the diagnostic, therapeutic and professional capabilities of a clinicians, which is especially important in the field of ophthalmic oncology, where there is a patient survival matter.

artifitial intellegence, machine learning, deep learning, neural networks, ophthalmology, ophthalmic oncology

1. Averkin A.N., Gaaze-Rapoport M.G., Pospelov D.A. Glossary on Artificial Intelligence. Moscow: Radio i svyaz’, 1992. P. 38–39 (In Russ.)

2. Pesapane F., Codari M., Sardanelli F. Artificial intelligence in medical imaging: threat or opportunity? Radiologists again at the forefront of innovation in medicine. European Radiology Experimental. 2018;2(1):35. DOI: 10.1186/s41747018-0061-6

3. Lakhani P., Prater A.B., Hutson R.K., Andriole K.P., Dreyer K.J., Morey J., Prevedello L.M., Clark T.J., Geis J.R., Itri J.N., Hawkins C. M. Machine Learning in Radiology: Applications Beyond Image Interpretation. Journal of the American College of Radiology. 2018;15(2):350–359. DOI: 10.1016/j.jacr.2017.09.044

4. Kappor R., Walters S.P., Al-Aswad L.A. The Current State of Artificial Intelligence in Ophthalmology. Survey of Ophthalmology. 2018; Sep 22. DOI: 10.1016/j.survophthal.2018.09.002

5. Samuel, A.L. Some Studies in Machine Learning Using the Game of Checkers. IBM Journal of Research and Development. 1959;3(3):210–229. DOI: 10.1147/rd.33.0210

6. Bishop C.M. Pattern recognition and machine learning. New York: Springer; 2006. P. 2–3.

7. Kotsiantis S.B. Supervised machine learning: a review of classification techniques. Informatica. 2007;31:249–268.

8. Liaw A., Wiener M. Classification and regression by random Forest. R news. 2002;2:18–22.

9. Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev. 1958;65:386.

10. King B.F. Guest Editorial: Discovery and Artificial Intelligence. American Journal of Roentgenology. 2017;209(6):1189–1190. DOI: 10.2214/ajr.17.19178

11. Deng L., Yu D. Deep Learning: Methods and Applications. Foundations and Trends in Signal Processing. 2014;7(3–4):1–199.

12. Chartrand G., Cheng P.M., Vorontsov E., Drozdzal M., Turcotte S., Pal C.J., Kadoury S., Tang A. Deep Learning: A Primer for Radiologists. RadioGraphics. 2017;37(7):2113–2131. DOI: 10.1148/rg.2017170077

13. Sánchez C.I., Niemeijer M., Dumitrescu A.V., Suttorp-Schulten M.S.A., Abràmoff M.D., van Ginneken B. Evaluation of a Computer-Aided Diagnosis System for Diabetic Retinopathy Screening on Public Data. Investigative Opthalmology & Visual Science. 2011;52(7):4866. DOI: 10.1167/iovs.10-6633

14. Marin D., Gegundez-Arias M.E., Suero A., Bravo J.M. Obtaining optic disc center and pixel region by automatic thresholding methods on morphologically processed fundus images. Computer Methods and Programs in Biomedicine. 2015;118(2):173– 185. DOI: 10.1016/j.cmpb.2014.11.003

15. Quellec G., Lamard M., Abràmoff M.D., Decencière E., Lay B., Erginay A., Cochener B., Cazuguel G. A multiple-instance learning framework for diabetic retinopathy screening. Medical Image Analysis. 2012;16(6):1228–1240. DOI: 10.1016/j. media.2012.06.003

16. Decencière E., Cazuguel G., Zhang X., Thibault G., Klein J.C., Meyer F., Marcotegui B., Quellec G., Lamard M., Danno R., Elie D., Massin P., Viktor Z., Erginay A., Laÿ B., Chabouis A. TeleOphta: Machine learning and image processing methods for teleophthalmology. IRBM. 2013;34(2):196–203. DOI: 10.1016/j.irbm.2013.01.010

17. Choi J.Y., Yoo T.K., Seo J.G., Kwak J., Um T.T., Rim T.H. Multi-categorical deep learning neural network to classify retinal images: A pilot study employing small database. PLOS ONE. 2017; 12(11):e0187336. DOI: 10.1371/journal.pone.0187336

18. Gulshan V., Peng L., Coram M., Stumpe M.C., Wu D., Narayanaswamy A., Venugopalan S., Widner K., Madams T., Cuadros J., Kim R., Raman R., Nelson P.C., Mega J.L., Webster, D.R. Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA. 2016;316(22):2402. DOI: 10.1001/jama.2016.17216

19. Gargeya R., Leng, T. Automated Identification of Diabetic Retinopathy Using Deep Learning. Ophthalmology. 2017;124(7):962–969. DOI: 10.1016/j.ophtha.2017.02.008

20. Abràmoff M.D., Folk, J.C., Han, D.P., Walker, J.D., Williams, D.F., Russell, S.R., Massin P., Cochener B., Gain P., Tang L., Lamard M., Moga D.C., Quellec G., Niemeijer M. Automated Analysis of Retinal Images for Detection of Referable Diabetic Retinopathy. JAMA Ophthalmology. 2013;131(3):351. DOI: 10.1001/jamaophthalmol.2013.1743

21. Abràmoff M.D., Lou Y., Erginay A., Clarida W., Amelon R., Folk J.C., Niemeijer M. Improved Automated Detection of Diabetic Retinopathy on a Publicly Available Dataset Through Integration of Deep Learning. Investigative Opthalmology & Visual Science. 2016;57(13):5200. DOI: 10.1167/iovs.16-19964

22. Takahashi H., Tampo H., Arai Y., Inoue Y., Kawashima H. Applying artificial intelligence to disease staging: Deep learning for improved staging of diabetic retinopathy. PLOS ONE. 2017;12(6):e0179790. DOI: 10.1371/journal.pone.0179790

23. Schmidt-Erfurth U., Sadeghipour A., Gerendas B.S., Waldstein S.M., Bogunović H. Artificial intelligence in retina. Progress in Retinal and Eye Research. 2018;67:1–29. DOI: 10.1016/j.preteyeres.2018.07.004

24. Lawrence M.G. The accuracy of digital-video retinal imaging to screen for diabetic retinopathy: an analysis of two digital-video retinal imaging systems using standard stereoscopic seven-field photography and dilated clinical examination as reference standards. Trans Am Ophthalmol Soc. 2004;102:321–340.

25. Tsao H.Y., Chan P.Y., Su E.C.Y. Predicting diabetic retinopathy and identifying interpretable biomedical features using machine learning algorithms. BMC Bioinformatics. 2018;19(S9):195–205. DOI: 10.1186/s12859-018-2277-0

26. Jiang Z., Yu Z., Feng S., Huang Z., Peng Y., Guo J., Ren Q., Lu Y. A super-resolution method-based pipeline for fundus fluorescein angiography imaging. BioMedical Engineering OnLine. 2018;17(1):125. DOI: 10.1186/s12938-018-0556-7

27. Serrano-Aguilar P., Abreu R., Antón-Canalís L., Guerra-Artal C., Ramallo-Fariña Y., Gómez-Ulla F., Nadal J. Development and validation of a computer-aided diagnostic tool to screen for age-related macular degeneration by optical coherence tomography. British Journal of Ophthalmology. 2011;96(4):503–507. DOI: 10.1136/ bjophthalmol-2011-300660

28. Venhuizen F.G., van Ginneken B., van Asten F., van Grinsven M.J.J.P., Fauser S., Hoyng C.B., Theelen T., Sánchez C.I. Automated Staging of Age-Related Macular Degeneration Using Optical Coherence Tomography. Investigative Opthalmology & Visual Science. 2017;58(4):2318. DOI: 10.1167/iovs.16-20541

29. Treder M., Lauermann J. L., Eter N. Automated detection of exudative age-related macular degeneration in spectral domain optical coherence tomography using deep learning. Graefe’s Archive for Clinical and Experimental Ophthalmology. 2017;256(2):259–265. DOI: 10.1007/s00417-017-3850-3

30. Burlina P.M., Joshi N., Pekala M., Pacheco K.D., Freund D.E., Bressler N.M. Automated Grading of Age-Related Macular Degeneration From Color Fundus Images Using Deep Convolutional Neural Networks. JAMA Ophthalmology. 2017;135(11):1170. DOI: 10.1001/jamaophthalmol.2017.3782

31. Lee C.S., Baughman D.M., Lee A.Y. Deep Learning Is Effective for Classifying Normal versus Age-Related Macular Degeneration OCT Images. Ophthalmology Retina. 2017;1(4):322–327. DOI: 10.1016/j.oret.2016.12.009

32. Rahimy E. Deep learning applications in ophthalmology. Current Opinion in Ophthalmology. 2018;29(3):254–260. DOI: 10.1097/icu.0000000000000470

33. Bogunovic H., Montuoro A., Baratsits M., Karantonis M.G., Waldstein S.M., Schlanitz F., Schmidt-Erfurth U. Machine Learning of the Progression of Intermediate Age-Related Macular Degeneration Based on OCT Imaging. Investigative Opthalmology & Visual Science. 2017;58(6):BIO141–BIO150. DOI: 10.1167/iovs.17-21789

34. Bogunovic H., Waldstein S.M., Schlegl T., Langs G., Sadeghipour A., Liu X., Gerendas B.S., Osborne A., Schmidt-Erfurth U. Prediction of Anti-VEGF Treatment Requirements in Neovascular AMD Using a Machine Learning Approach. Investigative Opthalmology & Visual Science. 2017;58(7):3240. DOI: 10.1167/iovs.16-21053

35. Rohm M., Tresp V., Müller M., Kern C., Manakov I., Weiss M., Sim D.A., Priglinger S., Keane P.A., Kortuem K. Predicting Visual Acuity by Using Machine Learning in Patients Treated for Neovascular Age-Related Macular Degeneration. Ophthalmology. 2018;125(7):1028–1036. DOI: 10.1016/j.ophtha.2017.12.034

36. Hogarty D.T., Mackey D.A., Hewitt A.W. Current state and future prospects of artificial intelligence in ophthalmology: a review. Clinical & Experimental Ophthalmology. 2018;Aug 28. DOI: 10.1111/ceo.13381

37. Yousefi S., Goldbaum M.H., Balasubramanian M., Jung T.P., Weinreb R.N., Medeiros F.A., Zangwill L.M., Liebmann J.M., Girkin C.A., Bowd C. Glaucoma Progression Detection Using Structural Retinal Nerve Fiber Layer Measurements and Functional Visual Field Points. IEEE Transactions on Biomedical Engineering. 2014;61(4):1143–1154. DOI: 10.1109/tbme.2013.2295605

38. Oh E., Yoo T.K., Hong S. Artificial Neural Network Approach for Differentiating Open-Angle Glaucoma From Glaucoma Suspect Without a Visual Field Test. Investigative Opthalmology & Visual Science. 2015;56(6):3957. DOI: 10.1167/ iovs.15-16805

39. Zhang Z., Yin F.S., Liu J., Wong W.K., Tan N.M., Lee B.H., Cheng J., Wong T.Y. ORIGA-light: An online retinal fundus image database for glaucoma analysis and research. Annual International Conference of the IEEE Engineering in Medicine and Biology. 2010; 2010:3065–3068. DOI: 10.1109/iembs.2010.5626137

40. Sng C.C., Fo L.L., Cheng C.Y., Allen J.C., He M., Krishnaswamy G., Nongpiur M.E., Friedman D.S., Wong T.Y., Aung T. Determinants of Anterior Chamber Depth: The Singapore Chinese Eye Study. Ophthalmology. 2012;119(6):1143–1150. DOI: 10.1016/j.ophtha.2012.01.011

41. Chen X., Xu Y., Wong D.W.K., Wong T.Y., Liu J. Glaucoma detection based on deep convolutional neural network. 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). 2015;2015:715–718. DOI: 10.1109/embc.2015.7318462

42. Raghavendra U., Fujita H., Bhandary S.V., Gudigar A., Tan J.H., Acharya U.R. Deep convolution neural network for accurate diagnosis of glaucoma using digital fundus images. Information Sciences. 2018; 441(1):41–49. DOI: 10.1016/j. ins.2018.01.051

43. Kim M., Zuallaert J., De Neve W. Few-shot Learning Using a Small-Sized Dataset of High-Resolution FUNDUS Images for Glaucoma Diagnosis. Proceedings of the 2nd International Workshop on Multimedia for Personal Health and Health Care — MMHealth, 2017. P. 89–92. DOI: 10.1145/3132635.3132650

44. Kim S.J., Cho K.J., Oh S. Development of machine learning models for diagnosis of glaucoma. PLOS ONE. 2017;12(5):e0177726. DOI: 10.1371/journal.pone.0177726

45. Souza M.B., Medeiros F.W., Souza D.B., Garcia R., Alves M.R. Evaluation of machine learning classifiers in keratoconus detection from orbscan II examinations. Clinics. 2010;65(12):1223–1228. DOI: 10.1590/s1807-59322010001200002

46. Arbelaez M.C., Versaci F., Vestri G., Barboni P., Savini G. Use of a Support Vector Machine for Keratoconus and Subclinical Keratoconus Detection by Topographic and Tomographic Data. Ophthalmology. 2012;119(11):2231–2238. DOI: 10.1016/j. ophtha.2012.06.005

47. Smadja D., Touboul D., Cohen A., Doveh E., Santhiago M.R., Mello G.R., Krueger R.R., Colin J. Detection of Subclinical Keratoconus Using an Automated Decision Tree Classification. American Journal of Ophthalmology. 2013;156(2):237– 246.e1. DOI: 10.1016/j.ajo.2013.03.034

48. Ruiz Hidalgo I., Rodriguez P., Rozema J.J., Ní Dhubhghaill S., Zakaria N., Tassignon M.J., Koppen C. Evaluation of a Machine-Learning Classifier for Keratoconus Detection Based on Scheimpflug Tomography. Cornea. 2016;35(6):827–832. DOI: 10.1097/ico.0000000000000834

49. Kovács I., Miháltz K., Kránitz K., Juhász É., Takács Á., Dienes L., Gergely R., Nagy Z. Z. Accuracy of machine learning classifiers using bilateral data from a Scheimpflug camera for identifying eyes with preclinical signs of keratoconus. Journal of Cataract & Refractive Surgery. 2016;42(2):275–283. DOI: 10.1016/j.jcrs.2015.09.020

50. Ambrósio R., Lopes B.T., Faria-Correia F., Salomão M.Q., Bühren J., Roberts C.J., Elsheikh A., Vinciguerra R., Vinciguerra P. Integration of Scheimpflug-Based Corneal Tomography and Biomechanical Assessments for Enhancing Ectasia Detection. Journal of Refractive Surgery. 2017;33(7):434–443. DOI: 10.3928/1081597x-20170426-02

51. Ting D.S.W., Cheung C. Y.L., Lim G., Tan G.S.W., Quang N.D., Gan A., Hamzah H., Garcia-Franco R., San Yeo I.Y., Lee S.Y., Wong E.Y.M., Sabanayagam C., Baskaran M., Ibrahim F., Tan N.C., Finkelstein E.A., Lamoureux E.L., Wong I.Y., Bressler N.M., Sivaprasad S., Varma R., Jonas J.B., He M.G., Cheng C.Y., Cheung G.C.M., Aung T., Hsu W., Lee M.L., Wong T.Y. Development and Validation of a Deep Learning System for Diabetic Retinopathy and Related Eye Diseases Using Retinal Images From Multiethnic Populations With Diabetes. JAMA. 2017;318(22):2211. DOI: 10.1001/jama.2017.18152

52. Kermany D.S., Goldbaum M., Cai W., Valentim C.C.S., Liang H., Baxter S.L., McKeown A., Yang G., Wu X., Yan F., Dong J., Prasadha M.K., Pei J., Ting M.Y.L., Zhu J., Li C., Hewett S., Dong J., Ziyar I., Shi A., Zhang R., Zheng L., Hou R., Shi W., Fu X., Duan Y., Huu V.A.N., Wen C., Zhang E.D., Zhang C.L., Li O., Wang X., Singer M.A., Sun X., Xu J., Tafreshi A., Lewis M.A., Xia H. Zhan, K. Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning. Cell. 2018;172(5):1122–1131.e9. DOI: 10.1016/j.cell.2018.02.010

53. Schlegl T., Waldstein S.M., Bogunovic H., Endstraßer F., Sadeghipour A., Philip A.M., Podkowinski D., Gerendas B.S., Langs G. Schmidt-Erfurth U. Fully Automated Detection and Quantification of Macular Fluid in OCT Using Deep Learning. Ophthalmology. 2018;125(4):549–558. DOI: 10.1016/j.ophtha.2017.10.031

54. Samagaio G., Estévez A., Moura J., Novo J., Fernández M.I., Ortega M. Automatic macular edema identification and characterization using OCT images. Computer Methods and Programs in Biomedicine. 2018;163:47–63. DOI: 10.1016/j. cmpb.2018.05.033

55. Gao X., Lin S., Wong T.Y. Automatic Feature Learning to Grade Nuclear Cataracts Based on Deep Learning. IEEE Transactions on Biomedical Engineering. 2015;62(11):2693–2701. DOI: 10.1109/tbme.2015.2444389

56. Liu X., Jiang J., Zhang K., Long E., Cui J., Zhu M., An Y., Zhang J., Liu Z., Lin Z., Li X., Chen J., Cao Q., Li J., Wu X., Wang D., Li H. Localization and diagnosis framework for pediatric cataracts based on slit-lamp images using deep features of a convolutional neural network. PLOS ONE. 2017;12(3):e0168606. DOI: 10.1371/ journal.pone.0168606

57. Zhang, Li J., Zhang I., Han H., Liu B., Yang J., Wang Q. Automatic cataract detection and grading using Deep Convolutional Neural Network. IEEE 14th International Conference on Networking, Sensing and Control (ICNSC), 2017. P. 60–65. DOI: 10.1109/icnsc.2017.8000068

58. Kaiserman I., Rosner M., Pe’er J. Forecasting the Prognosis of Choroidal Melanoma with an Artificial Neural Network. Ophthalmology. 2005;112(9):1608. DOI: 10.1016/j.ophtha.2005.04.008

59. Damato B., Eleuteri A., Fisher A.C., Coupland S.E., Taktak A.F.G. Artificial Neural Networks Estimating Survival Probability after Treatment of Choroidal Melanoma. Ophthalmology. 2008;115(9):1598–1607. DOI: 10.1016/j.ophtha.2008.01.032

60. Vaquero-Garcia J., Lalonde E., Ewens K.G., Ebrahimzadeh J., Richard-Yutz J., Shields C. L., Barrera A., Green C.J., Barash Y., Ganguly A. PRiMeUM: A Model for Predicting Risk of Metastasis in Uveal Melanoma. Investigative Opthalmology & Visual Science. 2017;58(10):4096. DOI: 10.1167/iovs.17-22255

61. Khosravi, P., Kazemi, E., Imielinski, M., Elemento, O., & Hajirasouliha, I. Deep Convolutional Neural Networks Enable Discrimination of Heterogeneous Digital Pathology Images. EBioMedicine. 2018;27:317–328. DOI: 10.1016/j.ebiom.2017.12.026

62. Kwak J.T., Hewitt S.M., Sinha S., Bhargava, R. Multimodal microscopy for automated histologic analysis of prostate cancer. BMC Cancer. 2011;11(1):1–16. DOI: 10.1186/1471-2407-11-62

63. Hamilton P.W., Wang Y., Boyd C., James J.A., Loughrey M.B., Hougton, J.P., Boyle D.P., Kelly P. , Maxwell P., McCleary D., Diamond J., McArt DG., Tunstall J., Bankhead P., Salto-Tellez M. Automated tumor analysis for molecular profiling in lung cancer. Oncotarget. 2015;6(29):27938–27952 DOI: 10.18632/oncotarget.4391

64. Wang L.W., Qu A.P., Yuan J.P., Chen C., Sun S.R., Hu M.B., Liu J., Li Y. ComputerBased Image Studies on Tumor Nests Mathematical Features of Breast Cancer and Their Clinical Prognostic Value. PLoS ONE. 2013;8(12):e82314. DOI: 10.1371/ journal.pone.0082314

65. Bruno K., Andrea M.O., Allen P.M., Catherine M.N., Matthew A.S., Lorenzo T., Arief A.S., Saeed H.Deep learning for classification of colorectal polyps on wholeslide images. PLoS One. 2013;8(12):e82314.

66. Janowczyk A., Chandran S., Singh R., Sasaroli D., Coukos G., Feldman M.D., Madabhushi A. High-Throughput Biomarker Segmentation on Ovarian Cancer Tissue Microarrays via Hierarchical Normalized Cuts. IEEE Transactions on Biomedical Engineering. 2012;59(5):1240–1252. DOI: 10.1109/tbme.2011.2179546