Artificial intelligence 7: Language models for optometry

Artificial Intelligence (AI) has generated great interest within the healthcare industry. From research to clinical care and education, AI has the potential to change how we deliver better healthcare to everyone.  

For eye care professionals (ECPs), this means having access to advanced tools to improve prevention and/or management of eye conditions.  

For educators and learners, AI can offer personalised support, which can lead to enhanced learning outcomes and experience. For eye care businesses, AI can help enhance customer experiences and streamline operations.  

Many healthcare professionals, including optometrists, may not (yet) understand the possibilities of AI. AI is a branch of computer science focused on creating systems or machines that can perform tasks that typically require human intelligence. These tasks include learning, reasoning, problem-solving, perception and understanding natural language.  

This article explores how AI, especially in the context of Large Language Models (LLMs), can improve the work of clinical care and optical businesses, particularly focused at what is available today and what are the possibilities for the future. 

AI in clinical practice  

Every so often, new technology emerges that changes our daily practice of optometry. For instance, the introduction of optical coherence tomography (OCT) significantly enhanced our ability to diagnose and manage eye diseases.¹ Similarly, any AI tool designed to improve diagnostic accuracy and patient management could greatly benefit the field.  

For example, a recent study showed that AI-based detection achieved a higher level of accuracy than retinal specialists observing OCT scans when trying to detect retinal fluid.² Such advancements could alleviate the existing burden on ophthalmology and increase efficiency,^3,4 ultimately leading to economic benefits.⁵  

As the incidence of eye disease continues to rise,^6,7 automation presents a viable solution to meet the growing demand. Automation has the potential to boost productivity across various aspects of optometric care, by rapidly analysing large datasets, providing insights and offering personalised treatment recommendations. 

Large Language Models (figure 1) are one of the most utilised areas of AI today. They are a type of AI designed to understand and generate human language. They are based on deep learning techniques, particularly neural networks, that mimic the human brain’s structure and function.  

^{Figure 1: An illustration demonstrating how LLMs fit into the broader artificial intelligence environment and several examples of commercial use}

When it comes to eye disease, their effectiveness has been studied extensively.^8-11 This includes its use in providing patient advice based on symptom information,^12,13 generating clinical reports¹⁴ and directing patients to the right specialists based on their medical needs.¹⁵  

These AI tools can quickly analyse vast amounts of clinical and research data, more than any one professional could do in a traditional clinical appointment, identifying cases that need immediate attention. Combining this data with an oversight of an eye care professional’s opinion would amplify the effectiveness of these AI tools.  

The most widely used LLM tool today is ChatGPT, an AI chatbot that can be accessed on a browser or mobile phone application. ChatGPT is trained on massive datasets comprising text from books, articles, websites and other written material.¹⁶ Studies show it can offer clinical guidance as accurately as an ECP.^{12, 13}  

For example, a patient could ask what action to take following symptoms of a red or painful eye. To save time during the appointment, a clinician could input a series of clinical history, signs, symptoms, lifestyle and medication information to offer personalised guidance to a patient or even write referral letters. Consequently, this frees up time, which clinicians could dedicate towards communication and patient care.  

While LLM tools have many benefits for the health care professional, it is also important to note the challenges we face in using these models, especially when it could pose a risk to patient health. The accuracy of ChatGPT’s responses depends on the quality of the data it was trained on. If the training data includes incorrect or misleading information, the model might propagate these errors. This would lead to incorrect recommendations, potentially delaying necessary medical treatment.  

Healthcare, including eye care, evolves rapidly and keeping the model updated with the latest research, guidelines and best practices is challenging since the training data may become outdated.  

Furthermore, clinical management guidelines differ geographically and therefore it is important that this is considered when clinicians use ChatGPT for clinical recommendations. This can be done by uploading any relevant guidelines or other documentation within the chat itself for the model to refer to when generating responses.  

In addition, unlike human experts, ChatGPT cannot easily cross-check or validate the information it provides, making it harder to identify and correct errors. However, ChatGPT and other language models can be trained on verified information and instructed to cite its sources within custom built applications.  

^{Figure 2: An example of how a clinician could utilise Aeyeconsult where sources of information can be displayed (answer to clinician query has been excluded in the diagram)}

An example of this is Aeyeconsult,¹⁷ a generative AI chatbot built to answer eye care related questions (figure 2). It does this by using GPT technology trained on only verified ophthalmology textbooks as data and it cites its sources. Some examples of questions a healthcare provider could ask are: 

“What should I consider when diagnosing a 35-year-old patient who has headaches, problems with colour vision recently in the right eye and frequent headaches. VA in this eye is 6/6 and patient has been taking oral steroids recently.”  

“What is the possible differential diagnosis for unilateral red eye with mild pain and light sensitivity. Give me an indication of what tests I could perform to make a more informed decision” 

A study showed that on the Ophthalmology Knowledge Assessment Program (OKAP) examination dataset, Aeyeconsult outperformed ChatGPT-4, with 83.4% correct answers compared to 69.2%, respectively.¹⁷ This shows that language models can be much more effective in its responses when trained on verified, credible sources. 

Future of AI in clinical practice 

In the future, more sophisticated LLMs could assist ECPs in their management tasks, allowing them to dedicate more time to patients who require the most care. These LLMs would be able to analyse clinical images, medical records, lifestyle data from digital devices, genetic data, trusted papers and expert opinion on similar cases.  

Subsequently, it could flag any areas of concern to an ECP. This could all be done within a few seconds, much faster and more consistent, outperforming any human healthcare professional. This vision has shown some promise by health researchers at Google who developed Med-PaLM 2 ¹⁸ a language model specifically designed for medical applications.  

To show its state-of-the-art effectiveness, Med-PaLM 2 was assessed against a benchmark called ‘MultiMedQA’, which combines seven question-answering datasets spanning professional medical exams, medical research and consumer queries.¹⁹ It achieved a passing score on the United States Medical Licensing Examination (USMLE) of 86.5%.  

It has been extended beyond language, to integrate visual detection capabilities so that it can analyse clinical images. This feature has shown tremendous progress in radiology.²⁰ In the future, ECPs and patients could speak to an artificial assistant to guide them in the management of eye conditions. 

The above models are classed as foundation models, which are large-scale machine learning models that are trained on a broad and diverse range of data, enabling them to perform a wide array of tasks with minimal task-specific adjustments.  

An example of a model specifically designed for ophthalmology is RetFound,²¹ a cutting-edge self-supervised foundation model developed by ophthalmologists at Moorfields Eye Hospital and data engineers at Google Deepmind.  

RetFound utilises advanced machine learning techniques to analyse retinal images. Models such as RetFound hold significant promise for the future of optometry, offering a powerful tool to enhance diagnostic accuracy and efficiency. By automatically identifying patterns and anomalies in retinal images, it can aid in the early detection of conditions such as diabetic retinopathy, glaucoma and age-related macular degeneration, often before symptoms become apparent. This capability would allow optometrists to initiate timely interventions and personalised treatment plans, improving patient outcomes. 

Additionally, RetFound can streamline routine screening processes, freeing up valuable time for optometrists to focus on coaching the patient’s overall health journey, managing complex cases and offering treatment. This would facilitate a more holistic approach to providing care from the test room.  

However, reaching this stage would require overcoming several barriers.²¹ These include more research into the model’s accuracy across global datasets, different demographic profiles and imaging devices. Other factors, such as access to required computing power across institutions, could be a challenge due to financial constraints.  

On a regulatory level, rigorous validation and approval by relevant health authorities are required before any model such as RETFound can be used in clinical settings. It must comply with ethical guidelines and legal regulations concerning patient data privacy and the use of AI in medical decision-making. A digital health strategy that addresses this, plus outlining new roles and responsibilities for optometrists could help advance the optometry profession through such technology. 

A common problem reported by ECPs is the limited time available to provide the full requirements of an eye test among a full day’s clinic, especially for complex cases. 

However, LLMs can transcribe patient and clinician voice interaction into text²² and log it for review before including them in clinical records. This frees up more quality time to be spent on treating and advising patients.  

If a universal digital health system is in place, the same technology would be able to write a referral, check ophthalmology clinic diaries and even arrange them based on urgency of intervention. 

AI in Education 

Everyone learns at a different pace, using a variation of skills, and within their preferred environment. Therefore, to maximise learning outcomes, each student’s unique needs should be addressed. This requires a more personalised approach to teaching each individual. Whether you are an optometry educator or learner, an early career or expert, LLMs can support your goal.  

For example, an educator can use ChatGPT to create tailored lesson plans based on a student’s existing capabilities.²³ The same tool can present various scenarios, gather responses and then identify gaps in a student’s knowledge and provide further prompts to fill the gap.²⁴  

These scenarios could be formed with a trained knowledge base from a trusted source such as the College of Optometrists clinical management guidelines combined with the latest scientific literature on a certain eye condition, all uploaded within a ChatGPT custom prompt.  

In addition, GOC requirements could then assess if the user does or does not meet the requirements for the relevant competencies of interest.  

An educational iOS app called Ask Fellow Optoms (with which author KD is involved) has recently been launched with AI capabilities to help optometrists improve their confidence with patient management (figure 3).  

^{Figure 3: Ask Fellow Clinicians App illustrating (a) a clinician query with a ‘see AI response’ tab within a community chat, (b) GPT-4o large language model guidance to query shared (founded by author KD)}

It allows optometrists to discuss eye care queries among each other to improve learning and patient management through clinician experiences. In addition to obtaining a fellow clinicians’ opinion within the app, a supplementary ‘AI response’ is being tested. When a user shares a case, it offers areas of consideration to aid learning (without offering a diagnosis or management plan). To do this, it integrates the GPT-4o model, which has superior precision in its responses than the GPT-4 model.  

The effectiveness of this model to answer clinical queries has been demonstrated in various studies across healthcare, showing promising results. Various models are being experimented.  

In the future, as these models get better at analysing images and text it could provide better access to quality optometry education. 

Rather than learning through reading literature LLMs integrated with virtual reality (VR) can facilitate real-world interactive simulations and virtual patient interactions, enhancing practical learning experiences without the need for physical presence.²⁵ This would make high quality education accessible to all students. 

Outside of clinician education, LLMs can also support patient education. The Gene Vision chatbot improves access of resources for visually impaired individuals who have genetic eye disease.^{26, 27} Accessible through Alexa voice and a chat interface, users can ask specific questions such as ‘how is my condition inherited?’ or ‘what employment help is available?’  

AI in business operations 

Customer expectations are increasing, especially within the healthcare industry. Customers want answers to their questions and solutions to their problems at the click of their finger.  

A model such as GPT-4 or Google Gemini could be trained on all in-house procedures and local clinical protocols, so that when a customer interacts with the business website, they could receive the correct information at speed without the need for a trained member of staff.  

Interestingly, the LLM could also integrate with (part of) the business database to provide more personalised experiences for the customer, for example to check if their spectacles are ready or when their next eye examination or contact lens aftercare is due.  

ChatGPT can also be used by optical businesses to streamline common operations to accentuate customer satisfaction, improve customer retention and, ultimately, increase revenue.²⁸  

When it comes to staff training, ChatGPT can develop training modules and resources for staff to stay updated on the latest products, technologies and customer service techniques based on practice values and goals.  

Also, consumer relationships are built over time through trust. Communication through social and email marketing content can improve trust in optical business brands. Practices that struggle to keep up with marketing content, may wish to use ChatGPT to generate ideas and produce content for social media channels to enhance marketing efforts, saving hours of manual work and overhead costs.²⁹   

Summary 

In conclusion, the integration of AI, particularly through Large Language Models, offers transformative potential across the eye care sector. By enhancing clinical practices, personalising education and optimising business operations, AI can significantly improve efficiency and efficacy.  

As AI continues to evolve, its adoption could lead to better patient outcomes, advanced and tailored educational experiences, and more responsive eye care businesses. 

However, challenges such as data accuracy and model updating need careful management to fully realise AI’s benefits in eye care. This technological advancement seems to be a promising future for the healthcare industry, provided it is implemented thoughtfully and ethically.  

• ^{Kishan Devraj is a qualified optometrist in the UK. He has been awarded a PhD studentship at UCL Institute of ophthalmology where he focuses on harnessing smartphone technology to assess the lifestyle impact and quality of life with eye disease. He is an honorary clinical research fellow at Moorfields Eye Hospital. His contribution in building the OverSight app helped him win the Clinical Leader of the Future Award (Optician, 2023) and achieve a place on the NHS Clinical Entrepreneur programme. He is also founder of Ask Fellow Optoms.} 

• ^{Dr Byki Huntjens is a qualified optometrist from the Netherlands who obtained her MSc and PhD at the University of Manchester in the UK. She taught contact lenses at City, University of London for over a decade and recently became an independent education and research consultant in contact lenses and dry eye. In addition, Huntjens is educational lead with the Association of Optometrists, and an honorary senior research fellow (City University) collaborating on TFOS Lifestyle: digital environment report, BCLA CLEAR, and Cochrane review on ‘Interventions for myopia control in children’. She is a council member and President-Elect of the BCLA, and director of IACLE.}  

References 

1. Wang Y, Liu S, Lou S, Zhang W, Cai H, Chen X. Application of optical coherence tomography in clinical diagnosis. J Xray Sci Technol. 2019 Jan 1;27(6):995–1006.  
2. Keenan TDL, Clemons TE, Domalpally A, Elman MJ, Havilio M, Agrón E, et al. Retinal Specialist versus Artificial Intelligence Detection of Retinal Fluid from OCT: Age-Related Eye Disease Study 2: 10-Year Follow-On Study. Ophthalmology. 2021 Jan 1;128(1):100–9. 
3. Ramessur R, Raja L, Kilduff CLS, Kang S, Li JPO, Thomas PBM, et al. Impact and Challenges of Integrating Artificial Intelligence and Telemedicine into Clinical Ophthalmology. Asia-Pacific Journal of Ophthalmology. 2021 May 1;10(3):317–27. 
4. Alowais SA, Alghamdi SS, Alsuhebany N, Alqahtani T, Alshaya AI, Almohareb SN, et al. Revolutionizing healthcare: the role of artificial intelligence in clinical practice. BMC Medical Education 2023 23:1 [Internet]. 2023 Sep 22 [cited 2024 Jul 17];23(1):1–15. Available from: https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-023-04698-z
5. Xie Y, Nguyen QD, Hamzah H, Lim G, Bellemo V, Gunasekeran D V, et al. Artificial intelligence for teleophthalmology-based diabetic retinopathy screening in a national programme: an economic analysis modelling study. Lancet Digit Health [Internet]. 2020 May 1 [cited 2024 Jul 18];2(5):e240–9. Available from: http://www.thelancet.com/article/S2589750020300601/fulltext
6. Li JQ, Welchowski T, Schmid M, Letow J, Wolpers C, Pascual-Camps I, et al. Prevalence, incidence and future projection of diabetic eye disease in Europe: a systematic review and meta-analysis. Eur J Epidemiol [Internet]. 2020 Jan 1 [cited 2024 Jul 18];35(1):11–23. Available from: https://link.springer.com/article/10.1007/s10654-019-00560-z
7. Li JQ, Welchowski T, Schmid M, Mauschitz MM, Holz FG, Finger RP. Prevalence and incidence of age-related macular degeneration in Europe: a systematic review and meta-analysis. British Journal of Ophthalmology [Internet]. 2020 Aug 1 [cited 2024 Jul 18];104(8):1077–84. Available from: https://bjo.bmj.com/content/104/8/1077
8. Tan TF, Thirunavukarasu AJ, Campbell JP, Keane PA, Pasquale LR, Abramoff MD, et al. Generative Artificial Intelligence Through ChatGPT and Other Large Language Models in Ophthalmology: Clinical Applications and Challenges. Ophthalmology Science. 2023 Dec 1;3(4):100394. 
9. Betzler BK, Chen H, Cheng CY, Lee CS, Ning G, Song SJ, et al. Large language models and their impact in ophthalmology. Lancet Digit Health [Internet]. 2023 Dec 1 [cited 2024 Jul 18];5(12):e917–24. Available from: http://www.thelancet.com/article/S2589750023002017/fulltext
10. Zhou Y, Chia MA, Wagner SK, Ayhan MS, Williamson DJ, Struyven RR, et al. A foundation model for generalizable disease detection from retinal images. Nature 2023 622:7981 [Internet]. 2023 Sep 13 [cited 2024 Jul 18];622(7981):156–63. Available from: https://www.nature.com/articles/s41586-023-06555-x
11. Lim ZW, Pushpanathan K, Yew SME, Lai Y, Sun CH, Lam JSH, et al. Benchmarking large language models’ performances for myopia care: a comparative analysis of ChatGPT-3.5, ChatGPT-4.0, and Google Bard. EBioMedicine [Internet]. 2023 Sep 1 [cited 2024 Jul 18];95:104770. Available from: http://www.thelancet.com/article/S2352396423003365/fulltext
12. Pushpanathan K, Lim ZW, Er Yew SM, Chen DZ, Hui’En Lin HA, Lin Goh JH, et al. Popular large language model chatbots’ accuracy, comprehensiveness, and self-awareness in answering ocular symptom queries. iScience [Internet]. 2023 Nov 17 [cited 2024 Jul 18];26(11):108163. Available from: http://www.cell.com/article/S258900422302240X/fulltext
13. Bernstein IA, Zhang YV, Govil D, Majid I, Chang RT, Sun Y, et al. Comparison of Ophthalmologist and Large Language Model Chatbot Responses to Online Patient Eye Care Questions. JAMA Netw Open [Internet]. 2023 Aug 1 [cited 2024 Jul 18];6(8):e2330320–e2330320. Available from: https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2808557
14. Leonardi G, Portinale L, Santomauro A. Enhancing Medical Image Report Generation through Standard Language Models: Leveraging the Power of LLMs in Healthcare. 2023 [cited 2024 Jul 18]; Available from: http://ceur-ws.org
15. Li Z, Wang L, Wu X, Jiang J, Qiang W, Xie H, et al. Artificial intelligence in ophthalmology: The path to the real-world clinic. Cell Rep Med [Internet]. 2023 Jul 18 [cited 2024 Jul 18];4(7):101095. Available from: http://www.cell.com/article/S2666379123002148/fulltext
16. Singh SK, Kumar S, Mehra PS. Chat GPT & Google Bard AI: A Review. 2023 International Conference on IoT, Communication and Automation Technology, ICICAT 2023. 2023; 
17. Singer MB, Fu JJ, Chow J, Teng CC. Development and Evaluation of Aeyeconsult: A Novel Ophthalmology Chatbot Leveraging Verified Textbook Knowledge and GPT-4. J Surg Educ. 2024 Mar 1;81(3):438–43. 
18. Med-PaLM: A Medical Large Language Model - Google Research [Internet]. [cited 2024 Jul 18]. Available from: https://sites.research.google/med-palm/
19. Singhal K, Tu T, Gottweis J, Sayres R, Wulczyn E, Hou L, et al. Towards Expert-Level Medical Question Answering with Large Language Models. 2023 May 16 [cited 2024 Jul 18]; Available from: https://arxiv.org/abs/2305.09617v1
20. Kim S, Lee CK, Kim SS. Large Language Models: A Guide for Radiologists. Korean J Radiol [Internet]. 2024 Feb 1 [cited 2024 Jul 18];25(2):126. Available from: /pmc/articles/PMC10831297/
21. Zhou Y, Chia MA, Wagner SK, Ayhan MS, Williamson DJ, Struyven RR, et al. A foundation model for generalizable disease detection from retinal images. Nature 2023 622:7981 [Internet]. 2023 Sep 13 [cited 2024 Jul 18];622(7981):156–63. Available from: https://www.nature.com/articles/s41586-023-06555-x
22. Blackley S V., Huynh J, Wang L, Korach Z, Zhou L. Speech recognition for clinical documentation from 1990 to 2018: a systematic review. J Am Med Inform Assoc [Internet]. 2019 Feb 11 [cited 2024 Jul 18];26(4):324–38. Available from: https://pubmed.ncbi.nlm.nih.gov/30753666/
23. van den Berg G, du Plessis E. ChatGPT and Generative AI: Possibilities for Its Contribution to Lesson Planning, Critical Thinking and Openness in Teacher Education. Education Sciences 2023, Vol 13, Page 998 [Internet]. 2023 Sep 29 [cited 2024 Jul 18];13(10):998. Available from: https://www.mdpi.com/2227-7102/13/10/998/htm
24. Hoffman R. Impromptu Amplifying Our Humanity Through AI. 2023; 
25. Kapadia N, Gokhale S, Nepomuceno A, Cheng W, Bothwell S, Mathews M, et al. Evaluation of Large Language Model Generated Dialogues for an AI Based VR Nurse Training Simulator. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) [Internet]. 2024 [cited 2024 Jul 18];14706 LNCS:200–12. Available from: https://link.springer.com/chapter/10.1007/978-3-031-61041-7_13
26. Gene Vision launches chatbot and Alexa service [Internet]. [cited 2024 Jul 18]. Available from: https://www.aop.org.uk/ot/science-and-vision/technology/2021/07/12/gene-vision-launches-chatbot-and-alexa-service
27. Chatbot – Gene Vision [Internet]. [cited 2024 Jul 18]. Available from: https://gene.vision/chatbot/
28. Raj R, Singh A, Kumar V, Verma P. Analyzing the potential benefits and use cases of ChatGPT as a tool for improving the efficiency and effectiveness of business operations. BenchCouncil Transactions on Benchmarks, Standards and Evaluations. 2023 Sep 1;3(3):100140. 
29. Reyes LNE, Villaverde DB. Management of Industry 4.0: ChatGPT as Digital Marketing Tool. 2024 [cited 2024 Jul 18];209–32. Available from: https://link.springer.com/chapter/10.1007/978-3-031-54485-9_9