The healthcare industry treats artificial intelligence like just another software rollout. We write policies, form committees, and assume standard guardrails will keep operations safe. But agentic AI behaves entirely differently than previous digital health tools.
At the recent eHealth26 conference in Halifax, Nova Scotia, I sat down with Julia Zarb. She is the force behind Blue x Blue, a company building AI infrastructure with governance incorporated. She warned that applying the old IT playbook to this new medium creates massive unseen risks.
Core Insight: Human Oversight is a Liability Trap Slapping a “human-in-the-loop” label onto an AI tool does not guarantee patient safety, it simply shifts the legal and operational liability onto overwhelmed clinicians.
The Human-in-the-Loop Fallacy
Executives love hearing that a human will review AI outputs before finalizing a clinical decision. In reality, these reviewers lack the time and system explainability to catch microscopic errors under high stress.
“What we’re doing is transferring the liability that comes with that review,” Zarb noted. “We don’t have what’s called explainability to go back in and see how that got to it, who knew what, when, and where, and why?”
The Threat of Invisible Errors
AI models fail in tiny, logical steps that easily bypass human reviewers.
“What makes agentic AI good is also its internal flaw,” Zarb explained. “The way that it processes, it carries forward little micro fissures and flaws.”
Standard IT guardrails fail because AI naturally jumps boundaries. When multiple AI agents interact, a single logical error propagates until it results in a much larger failure.
“If 100 agents speak to 100 other agents and those mistakes are brought forward, these aren’t mistakes that are just errors, these are mistakes that make sense [to Ai at the time] because of the way the AI is built.”
Questions Healthcare IT Leaders are Asking
Why is the “human-in-the-loop” approach risky for healthcare AI? Relying solely on human oversight transfers immense liability to clinical staff. Fast-paced environments and high stress prevent humans from properly auditing AI outputs. Furthermore, clinicians lack the technical explainability tools required to understand exactly how the AI arrived at its conclusions.
How do AI errors differ from standard digital health software bugs? Traditional software bugs are often clear and trackable. AI generates “micro fissures” or small gap-fillers that look perfectly logical to the system. These tiny errors bypass standard guardrails and multiply rapidly when different AI agents share data across an organization.
Learn more about Blue x Blue at https://www.bluexblue.com/
Listen and subscribe to the Healthcare IT Today Interviews Podcast to hear all the latest insights from experts in healthcare IT.
And for an exclusive look at our top stories, subscribe to our newsletter and YouTube.
Tell us what you think. Contact us here or on Twitter at @hcitoday. And if you’re interested in advertising with us, check out our various advertising packages and request our Media Kit.
About Brian Overstreet