Article Summary
Agentic AI—autonomous systems that act independently toward goals without human oversight at each step—is already deployed at scale in health systems worldwide, including patient scheduling platforms in Finland and triage systems in Portugal. While these systems promise significant efficiency gains, experts emphasize that effective oversight requires pairing agents with human supervisors, robust risk frameworks, and specification-driven validation to catch failures that become more likely as AI systems grow more capable.
- Already deployed globally: Finland's BeeHealthy platform and Portugal's UpHill system handle scheduling, patient identification, triage, and consent autonomously at national scale.
- Multi-agent architecture: Effective systems use stacked layers—point solutions, foundation models, agents, and multi-agent systems—coordinated by supervisor agents and grounded in a semantic layer.
- Human oversight is essential: Experts agree every agent should be paired with a human; risk frameworks must define what an agent can access, how long it persists, and accountability when failures occur.
- Validation challenges grow with capability: Smarter agents can do more but fail in more ways; formal verification works for static images but breaks down for agents that retrieve external data and behave differently across runs.
- Specification-driven validation needed: Safety requires datasets built from clinical protocols and decision rules to construct edge cases, though validation will always lag behind rapidly evolving AI technology.
Agentic AI has become one of the biggest debates in radiology this year, and a dedicated session at the 2nd Annual Global AI Conference, hosted by the Royal College of Radiologists (RCR), examined why the term needs unpacking.
If you look at early large language models (LLMs) such as ChatGPT, one could say they were still in diapers. Agentic AI, however, just moved out and got a job and the car keys. It acts autonomously, planning and acting on its own toward a goal without human direction or oversight at each step.
Already running, not coming
"Agentic AI is already running at national scale," Dr. Rowland Illing, chief medical officer at Amazon Web Services (AWS) and a former interventional radiologist, told the audience.
One example is Finland, where multiple counties have adopted BeeHealthy, a citizen-facing platform that handles scheduling, identification, and consent without a human at any step. In Portugal, a system called UpHill has triaged more than 40,000 patients through the national health hotline, with nurses reviewing the agent's decisions to catch what it gets wrong.
Agents, but with a boss
Illing laid out the architecture behind these systems as a stack. At the base sit point solutions, task-specific tools bolted onto single steps of a workflow: scheduling AI, image acceleration, dictation.
Above that come multimodal foundation models that combine imaging with electronic medical record data and pathology. Agents sit above those, and multi-agent systems above agents again, typically coordinated by a supervisor agent that delegates to sub-agents with narrower functions and receives their output back.
None of it works, Illing said, without a semantic layer: a shared, machine-readable definition of an institution's own terminology, data models, and clinical relationships. Without it, agents interpret the same clinical concept differently across departments, let alone across hospitals. This is also the reason why deployments so far cluster inside single organizations rather than spanning networks.
Rewriting how systems get built
Beyond patient-facing tools, agentic systems are also rewriting how software itself gets built. Illing described a five-level scale for coding autonomy, from level zero, where a human writes every line, up to level four, fully autonomous engineering, where an agent writes, tests, and deploys code with no human touching it.
Most organizations, he said, sit around level one today. Synava, the company behind the European radiology platform medavis, is further along: a year ago, agentic coding inside the organization was at zero. Now, 100% of the company's code is written primarily by agents, deployed with 94% accuracy, and signed off by humans rather than built by them line by line, Illing reports.
Different levels of autonomy
One framework presented broke agent autonomy into five levels, running from a system that only observes, up to one that operates fully on its own. The top three levels, speakers agreed, are where oversight has to tighten.
A more cautious note came from Dr. Hatim Abdulhussein, chief executive officer of Health Innovation Kent Surrey Sussex, part of the NHS Health Innovation Network covering two health systems and around 20 hospitals across the southeast of England.
"Every agent should be paired with a human," he was clear. In his work Abdulhussein's team has built twelve dimensions of risk alongside the autonomy framework, covering how far an agent can reach, how long it persists, and what data it can touch.
It is an attempt to understand when an agent acts without a human checking every step, who answers for it when something goes wrong.
Where it can still go wrong
How agentic systems fail was demonstrated by Panagiotis Kouvaros, PhD, principal research scientist at Safe Intelligence. His central point runs against instinct: the smarter an agent gets, the more it can do, and the more it can do, the more ways there are for it to fail.
For image-based AI, Kouvaros's lab uses formal verification: a mathematical proof, not a sample estimate, that a model's output stays consistent across an entire defined range of input variations, brightness, contrast, geometric transforms, noise, around a given test point.
If the model holds, the method certifies it as robust. If not, it returns a concrete counterexample. Run across a full test set, this produces what Kouvaros called robust accuracy, always a stricter, lower figure than standard accuracy, and one that tends to fall further as models get larger and more capable.
That approach breaks down for agents. Formal verification depends on bounding a model's input space mathematically, which works for a static image but not for a system that retrieves external data, maintains context across a long-running task, and behaves differently from one run to the next.
In place of it, Kouvaros's lab uses specification-driven validation: a dataset built not just from input-output pairs but from clinical protocols, decision rules, and ontologies, then used to construct edge cases in the areas where an agent is most likely to fail.
When details change
He described a triage agent built to flag chest pain as high urgency. A textbook description of a heart attack gets flagged correctly. Change the wording, the kind of variation that comes naturally from patients with different vocabularies, and the same case can slide into a low-urgency muscular complaint. A human clinician catches that instantly. An agent trained on narrower language patterns might not.
Asked whether validation could ever keep pace with a technology moving this fast, Kouvaros said: "AI will always be one step ahead of validation. Validation catches up. AI moves again. I don't think it will run ahead indefinitely."
