New research from an international team of scientists led by University College London and Moorsfield Eye Hospital published in The Lancet Digital Health assessed the viability of using artificial intelligence (AI)—bespoke and code-free deep learning (CFDL) models—to identify neonatal patients for plus disease, a hallmark sign of retinopathy of prematurity (ROP) that is highly prognostic for sight loss.
Give me some background.
Following the landmark Early Treatment For Retinopathy of Prematurity study, standardized screening protocols and treatments for ROP were established.
Since then, the survival rate of premature neonates has increased.
However, with the widespread scarcity of pediatric ophthalmologists, there is a need for accessible screening techniques that could be used in healthcare settings with limited neonatal services.
Talk about the study.
In this retrospective cohort study, investigators used retinal images from 1,370 neonatal patients admitted to a neonatal unit at Homerton Hospital, London, between 2008 and 2018.
Clinicians used the Retcam Version 2 device (Natus Medical) on all infants who were either born at less than 32 weeks gestational age or had a birthweight of less than 1501 grams.
What about the CFDL model?
Pre-trained bespoke and automated CFDL models were developed to classify retinal images as either healthy, pre-plus disease, or plus disease. To train the CFDL program, investigators used 7,414 posterior pole images of babies.
To externally validate both models (conducted in the United Kingdom), researchers used datasets from Brazil, Egypt, and the United States.
Findings?
When discriminating between healthy vs. pre-plus or plus disease, the bespoke model had an area under the curve (AUC) of 0.986 (confidence interval [CI] 95%, 0.973-0.996) and the CFDL model had an AUC of 0.989 (0.979-0.997) on the internal test.
Both models generalized well to external validation test sets for discriminating healthy eyes from those with pre-plus or plus disease.
What else?
The CFDL platform was inferior to the bespoke model on differentiating pre-plus disease from healthy or plus disease in the US dataset.
Performance was also reduced when tested on the 3nethra neo imaging device (Forus Health).
Limitations?
Investigators did not explicitly collect individual-level data on potential confounders of model performance, such as sex and ethnicity.
Additionally, while this study utilized a relatively large dataset (from the UK) for training both the bespoke and CFDL models, the external validation datasets were relatively small.
It is unclear how many images make up a sufficient sample size for modeling plus disease, but most reports use several thousands of images.
Significance?
While the CFDL model requires further research and optimization, the research team outlined that “it provides a potential alternative for model development in settings where specialist data science expertise and access to high-performance computing resources are scarce.”
Expert input?
According to Dr. Siegfried K. Wagner, first author of the study: “We are now further validating our tool in multiple hospitals in the UK and are seeking to learn how people interact with AI’s outputs to understand how we could incorporate the tool into real-world clinical settings.”