A nurse operating a fundus camera, taking images of a patient’s retina.

Socio-environmental factors impact the performance of artificial intelligence (AI) screening models used to detect eye disease and should be considered when evaluating model accuracy, a recently published Google Health paper has revealed.

While deep learning algorithms hold promise for improving clinician workflows and patient outcomes, these gains have yet to be fully demonstrated in a real-world clinical setting. So Google Health researchers conducted field studies in 11 Thai eye clinics, examining how one deep-learning system is used in patient care by nurses to screen patients for diabetic retinopathy.

Over a period of eight months, the research team visited the clinics regularly, observing current eye-screening workflows, conducting interviews and analysing nurses’ expectations from their new AI-assisted screening process and post-deployment experiences. The fieldwork found several factors that influenced model use and performance including poor lighting, leading to ungradable images and user frustration; network connectivity issues, resulting in screening delays or worse, rescheduled appointments; and concerns regarding potential patient hardship (time, cost and travel) from on-the-spot system referral recommendations, prompting some nurses to discourage patient participation in the study, reported researchers.

“Our findings indicate that several socio-environmental factors impact model performance, nursing workflows and the patient experience,” said Emma Beede, Google Health user experience researcher. “Deploying an AI system by considering a diverse set of perspectives in the design and development process is just one part of introducing new health technology that requires human interaction. It's important to also study and incorporate real-life evaluations in the clinic and engage meaningfully with clinicians and patients before the technology is widely deployed. That’s how we can best inform improvements to the technology and how it is integrated into care to meet the needs of clinicians and patients.”

The open access paper, A human-centred evaluation of a deep learning system deployed in clinics for the detection of diabetic retinopathy, is available here: https://dl.acm.org/doi/fullHtml/10.1145/3313831.3376718