AI won out against non-specialist doctors at eye assessment and providing suitable advice, a new study has found.

The study pitted the AI model generative pre-trained transformer (GPT)-4 against non-specialist doctors across a range of eye problem-related questions, including extreme light sensitivity, decreased vision, lesions, itchy and painful eyes. The questions were derived from a textbook test for trainee eye doctors, which wasn’t available on the internet, said Dr Arun Thirunavukarasu, lead author and academic foundation doctor at Oxford University Hospitals NHS Foundation Trust.

The study, published in PLOS Digital Health, also found GPT-4 gained similar scores to trainee and expert eye doctors, though top-performing eye doctors scored higher. “We could realistically deploy AI in triaging patients with eye issues to decide which cases are emergencies that need to be seen by a specialist immediately, which can be seen by a GP and which don’t need treatment,” said Dr Thirunavukarasu.

While authors said specialised large language models are rapidly developing and show promise, another study, published in the British Journal of Ophthalmology, found the proficiency of GPT-4 in answering questions about complex clinical ophthalmology cases was improved by prompting, but did not surpass ophthalmology trainees.