Article Summary
Despite widespread clinical AI adoption, the healthcare field lacks reliable methods to prove economic value, with evaluation standards varying widely, most studies failing to account for implementation costs, and successful cases rarely scaling across different hospitals due to fragmented pathways and budget structures.
- Nearly all 31 radiology AI studies reviewed found their tools cost-effective, but fewer than half accounted for adoption costs and only one tracked long-term outcomes.
- Current evaluation gaps include small, unrepresentative accuracy studies, unclear care pathway impacts, and lack of head-to-head comparisons against standard care.
- Health economic cases built for national standards like NICE differ from hospital-specific business cases that depend on which budget lines costs and savings actually affect.
- Even proven AI tools require case rebuilding at each hospital because every trust runs clinical pathways differently enough that single national cost models rarely survive implementation.
- Experts suggest moving from cost-per-QALY metrics toward cost per correct diagnosis, which captures accuracy and volume without requiring years of patient follow-up data.
Despite all the enthusiasm around clinical AI, the field still lacks reliable ways to prove its economic value, speakers argued at the Royal College of Radiologists' Global AI Conference.
With so many tools now in practice, differentiation matters, as Dr Jennifer Dixon DBE, Chief Executive of the Health Foundation, pointed out in her opening remarks.
AI applications range from low-risk back-office automation to high-risk clinical decision support, and each needs a different kind of evaluation. Her critique of the current system: proper NIHR-style trials take years and serious funding, while faster local evaluations vary wildly in quality and rarely get published, so lessons learned at one hospital rarely reach the next.
Lucy Gregory, a health economist at Hardian Health, presented a systematic review of AI economic evaluations in radiology, funded by the European Commission. Nearly all 31 studies included found their AI tool cost-effective, but Gregory urged caution. Most were poorly reported, and fewer than half accounted for the cost of adopting the AI tool itself. Only two studies met a higher evidentiary bar, and just one tracked long-term outcomes, following patients for a median of 8.8 years, evidence Gregory said she'd never seen replicated anywhere else in AI radiology research.
She suggests moving past cost-per-QALY, the field's default metric, toward "cost per correct diagnosis", a measure that doesn't require years of downstream follow-up but still captures both accuracy and volume.
Tom Lawrence, data scientist at NICE, walked through NICE's two-track system: early value assessments for tools still building their evidence base, full technology appraisals for ones with an established case. Recurring evidence gaps, in his experience: small, unrepresentative accuracy studies, unclear impact on care pathways, and a persistent lack of head-to-head comparisons against standard of care.
One hospital at a time
Not all economic cases are the same, as Anna Barnes, PhD, director of medical physics at Guy's and St Thomas's, pointed out from her work running a live AI evaluation project across three cancer pathways.
A health economic case built for NICE, geared toward QALYs and system-wide value, isn't the same as the business case a hospital finance director needs, which comes down to which budget line the cost and the saving actually land in. "You have to know how much you cost now," she said, without that baseline, there's no way to tell whether an AI tool has genuinely moved the needle.
After six years running health technology assessments across the UK, Barnes has also found that every trust runs its pathways differently enough that a single national cost model rarely survives contact with a second hospital, meaning even a well-proven AI tool may need its case rebuilt, one hospital at a time.
AI in radiology might already be cost-effective in a narrow sense. Proving it, funding it, and scaling it across a fragmented system are three separate, unsolved problems.
