In 2016, a team led by Google researchers published a deep learning system for diabetic retinopathy in JAMA (Gulshan et al.). It was trained on 128,175 retinal photographs from EyePACS in the United States and three eye hospitals in India. On the EyePACS-1 validation set, at its high-sensitivity operating point, it detected 97.5 percent of referable retinopathy with 93.4 percent specificity.
Those are accuracy figures. They describe how often the system agreed with a panel of ophthalmologists about a photograph. They do not say how many patients kept their sight.
For a hospital group, a health program, or a ministry in a market with few specialists, that gap is the whole question. An AI tool can only change outcomes if a chain of steps holds: the image or test is taken, the tool can read it, the patient is told, the patient reaches treatment, and the treatment happens. This article looks at what the published evidence shows about each link, and what to measure before you scale a tool.
It is written for the people who decide whether to deploy these tools. Clinical decisions for individual patients belong to qualified clinicians and to national guidance.
Why the stakes differ where specialists are scarce
The World Health Organization uses a threshold of 4.45 doctors, nurses, and midwives per 1,000 people as an indicative minimum for reaching the health targets in the Sustainable Development Goals. A 2022 study in BMJ Global Health by Ahmat and colleagues found a density of 1.55 per 1,000 across the WHO African Region in 2018. Only four countries in the region, Seychelles, Namibia, Mauritius, and South Africa, were above 4.45.
Where no specialist will read the image, the realistic comparison for a screening tool is often screening with AI against no screening at all, not AI against a specialist. That raises the potential benefit. It also raises the cost of a tool that fails quietly, because there may be no one positioned to notice.
Diabetic retinopathy: from a good model to a working program
Thailand ran the kind of test a buyer needs: a prospective study inside its national screening program. Ruamviboonsuk and colleagues reported in The Lancet Digital Health in 2022 on 7,651 patients at nine primary care sites. For vision-threatening retinopathy, the system's sensitivity was 91.4 percent and its specificity 95.4 percent. Retina specialists reading the same images had a sensitivity of 84.8 percent and a specificity of 95.5 percent. The authors wrote that social and environmental factors and workflows must be considered when scaling such systems.
A companion study showed what those factors look like. In a CHI 2020 paper, Beede and colleagues observed the system in eleven Thai clinics. The abstract names a tension between the model's thresholds for image quality and the images a resource-constrained clinic produces. Google's summary of the work describes lighting that varied between clinics, and images with blur or dark areas that the system called ungradable. The original protocol referred every ungradable image to an ophthalmologist. The team changed it so that eye specialists reviewed those images with the patient's records first, which the post says reduced unnecessary travel, missed work, and anxiety.
Each rejected image is a patient who has to be re-imaged, referred, or asked to come back. That is part of the tool's real performance.
One more caution concerns reproducibility. An independent attempt to replicate the 2016 results using public data did not reach the original performance, and its authors pointed to insufficient detail in the published method. Treat a strong paper as a reason to evaluate a tool on your own patients.
Tuberculosis: the screening gap is the outcome gap
The WHO's Global Tuberculosis Report 2025 estimated 1.23 million TB deaths in 2024. About 8.3 million people were newly diagnosed that year, which the WHO said was 78 percent of the estimated number who fell ill. Roughly one in five people who fell ill was not reported as diagnosed.
That is where chest X-ray reading software sits. In its 2021 screening guideline, the WHO allowed computer-aided detection (CAD) software to be used in place of human readers to interpret digital chest X-rays for TB screening and triage in people aged 15 and older. Coverage of the guideline noted that a positive CAD result should be followed by a diagnostic test, and that the WHO provides a tool for calibrating the score threshold to the population being screened.
So the outcome depends on what happens after the software flags someone. Does the person get a confirmatory test the same day, or are they asked to travel? Does a positive test lead to treatment, and is treatment completed? A CAD tool that raises detection while referral completion stays low moves the bottleneck without removing it.
Malaria: high burden, same discipline
The WHO's World Malaria Report 2025 estimated about 610,000 malaria deaths in 2024. The WHO African Region accounted for about 95 percent of them, and children under five accounted for about three quarters of the deaths in that region.
Tools offered here range from software that reads blood-smear images to systems that support triage of children with fever. The same rule applies: ask for evidence from settings like yours, then measure the chain from test to treatment, not the read alone.
What to measure: the outcome chain
An illustrative example, with made-up numbers. A district screens 1,000 people with diabetes in a month. The camera produces 900 gradable images; 100 people must be re-imaged or referred without a result. The software flags 80 of the 900. Of those 80, 48 reach an eye clinic. Of those, 40 are treated.
In this example, the model could improve its accuracy by several points and the number treated would barely move. Raising referral attendance from 60 percent to 85 percent would add about 17 treated patients a month. Most of the outcome is decided outside the model.
Measure each step separately:
- Coverage. Of the people eligible, how many were screened?
- Usable inputs. What share of images or tests could the tool read, and what happened to the rest?
- Local accuracy. Sensitivity and specificity in your population, against a reference standard, at the threshold you actually use.
- Referral completion. Of positive screens, how many reached the next step, and how long did it take?
- Treatment. How many started treatment, and how many completed it?
- Outcome. Where it can be measured, the clinical result the program exists for.
Collect steps 1, 2, 4, and 5 for a month before the tool arrives. Without that baseline, you cannot tell whether the tool changed anything.
Governance before scale
In June 2021, the WHO published Ethics and governance of artificial intelligence for health with six guiding principles, including protecting autonomy, ensuring transparency, fostering accountability, and ensuring equity. They translate into practical requirements for a deployment.
- A named clinician or program lead owns the threshold and the decision to change it.
- Health workers know what the tool's output means and what to do when they disagree with it.
- Performance is monitored after launch, by site, so a drop at one clinic is visible.
- Patients are told when software is part of their screening.
Regulatory requirements for AI software differ by country, and in some markets the pathway is still being written. Ask the national regulator what applies before you sign, not after.
Questions for a vendor before a pilot
- Where has the tool been evaluated prospectively, in populations and clinics like ours? Can we read the full results?
- What share of inputs does it reject in real clinics, and what does it recommend for those patients?
- Does it work offline or on a weak connection?
- Who sets the operating threshold, and can we change it?
- What post-deployment monitoring does it provide, by site?
- What outcome data, beyond accuracy, has any deployment collected?
Pick one screening pathway, record its outcome chain for a month, and send these six questions to two vendors. The answers will tell you which tool to pilot, or whether the money should go to referral transport first.
GradeCircle builds and runs six products of its own: AfroSynergy, CallerBridge, HealthSquire, Heyzl, Lexfora, and LukeAPI, two of them in beta. The assumptions in our models come from operating those systems, not from a framework. See what we run.
If you are building or scaling in these markets, see how GradeCircle approaches AI implementation.
