In a diagnostic test accuracy study published in JAMA Ophthalmology, researchers assessed at-home monitoring tests to observe if they could detect the reactivation of neurovascular age-related macular degeneration (nAMD) sufficiently enough for implementation in clinical situations.
Give me some background first.
Wet AMD (nAMD) is the leading cause of visual impairment in older adults.
Note: A majority of patients with nAMD require several years of monitoring in hospital eye clinics, which can be a burden on patients.
Which leads to: Researchers in this study hypothesized that at-home tests could lessen the burden on patients and release clinical capacity.
Now, talk about the study.
Participants were selected from six National Health Service (NHS) eye clinics in the United Kingdom (UK)—including the Moorfields Eye Hospital in London and other (unmentioned) clinics—from August 21, 2018, to March 31, 2020.
The investigation included 297 patients who were asked to take tests weekly. A hospital follow-up was available for 259 participants (312 eyes).
The tests evaluated included:
- KeepSight Journal (KSJ)
- paper booklet
- MyVisionTrack (mVT)
- mobile app
- MultiBit app (MBT)
- mobile app
Note: The reference standard for the study was the detection of active nAMD by an ophthalmologist at hospital follow-up.
Talk more about these participants.
Patient demographics and criteria were as follows:
- Aged 50+
- Mean age 74.9
- Standard deviation 6.6
- 174 female (58.6%); 123 male (41.4%)
- Had at least one eye treated for active nAMD and undergone treatment for nAMD 6 to 42 months before study
Findings?
The study found that:
- Median home monitoring testing frequency was 3 times a month
- Interquartile range (IQR) 1-4
- Estimated area under receiver operating characteristic curve (AUROC) was less than 0.6 for all home monitoring tests
- Only the KSJ summary score (calculated by averaging raw scores accruing in the interval between visits) was associated with lesion activity
- Odds ratio, 3.48; 95% Confidence Interval (CI), 1.09-11.13; P = .04
Note regarding AUROC: AUROC is a metric that is utilized to evaluate a model’s ability to differentiate between cases and non-cases. A number closer to 1.00 indicates the model's ability to do so is strong. In this study, AUROCs were estimated first using participants’ baseline data to predict the reference.
Note regarding summary scores: A summary score combines several measures into one overall score
Limitations?
Test developers did not provide test thresholds.
Meaning: the thresholds could not be assigned a priori and the authors had to estimate the AUROCs.
Therefore: The researchers did not compare AUROCs for the tests due to their inadequate accuracy.
Additionally, sensitivity and specificity were reported based on the Youden index, which does not consider relative clinical and other implications of false positive and false negative misclassifications.
What else?
The investigators did not monitor patients’ clinical visits.
The result: Since participants most likely reported their subjective experience to their consultants, the reference standard may have been influenced.
The authors also noted there may have been variations in care between eye clinics and how ophthalmologists assessed disease acuity.
Final limitations?
The researchers compared visual function tests against a retinal imaging reference standard, which—while they noted it as closely reflecting clinical reality—may not compare the same pathological components. They added that visual acuity may have been a stronger comparison.
How did sample size come into play?
Lastly, the study had a small sample size.
Nearly one-third of the participants withdrew from the study over its duration, and the remaining individuals who continued to test and provide data may have been representative of a selected subset.
Expert opinion?
The authors stated that their results showed, "Implementing any of these evaluated tests, with ophthalmologists only reviewing test positives, would mean most active lesions were missed, risking unnecessary sight loss."
Take home.
None of the tests evaluated by the researchers were deemed suitable for implementation in a clinical setting.