< + > Clinical AI Needs Version Control, Not Just Validation

The following is a guest article by Dr. Michael T. Neely

Healthcare AI is often governed as though validation were a one-time gate: test the system, approve it, deploy it, and move on. That model breaks down when the software changes.

AI tools can be updated far more frequently than traditional clinical technology. A vendor may replace an underlying model, modify a prompt or retrieval layer, change thresholds, retrain on new data, alter workflow logic, or release a new version without changing the product name clinicians see. When that happens, an organization can still have a “validated” system on paper while a materially different system is operating in practice.

The governance question is therefore not only, “Was this AI validated?” It is, “Which version was validated, which version is running now, what changed, and what evidence tells us the change is still safe for this use?”

Validation Without Version Identity is Incomplete

Healthcare organizations already understand version control in other high-stakes domains. Medication formularies, clinical protocols, cybersecurity patches, laboratory methods, and medical devices are not managed as timeless objects. Changes are documented because a change can alter performance, risk, or downstream workflow.

AI should be treated the same way.

The FDA’s final guidance on predetermined change control plans for AI-enabled device software functions offers a useful regulatory example. It asks manufacturers to describe planned modifications, the methods used to develop and validate those modifications, and the impact of those changes. Not every healthcare AI tool is an FDA-regulated medical device, but the operational principle travels well: change should be anticipated, documented, assessed, and tied to evidence.

Health systems need an internal version record that identifies, at minimum, the vendor and product, deployed model or build, material configuration, approved use, implementation date, validation evidence, and responsible owner. For generative AI, the record may also need to capture prompt templates, retrieval sources, orchestration logic, and other components that meaningfully shape output.

Material Changes Need a Trigger

Not every software update requires a full revalidation. A cosmetic interface change is different from replacing the model that drafts a clinical summary. The goal is not bureaucracy for its own sake; it is a defensible material-change process.

Organizations should define in advance which changes trigger review. Examples include a new model family, altered training or tuning, changes to decision thresholds, new data sources, expanded intended use, changes in clinical workflow, a new autonomous capability, or modifications that affect how outputs are presented to clinicians or patients.

Vendor contracts should support this process. A health system cannot govern a system it is not told has changed. Material-change notification should therefore be treated as an operational requirement, not simply a courtesy from the vendor.

Revalidation Should Match the Risk

Once a material change is identified, the next question is how much testing is enough. The answer should depend on what the system does and the consequences of error.

A documentation assistant may require testing for factual accuracy, omission, workflow fit, and clinician review burden. A predictive decision-support tool may require renewed performance assessment, subgroup analysis, calibration checks, or workflow-specific validation. A patient-facing system may also need testing for clarity, escalation behavior, unsafe advice, and whether human review remains available when needed.

The NIST AI Risk Management Framework is useful here because it treats risk management as a lifecycle activity rather than a single approval event. That lifecycle view is especially important for AI systems that can drift, be reconfigured, or change through vendor updates.

Keep the Ability to Stop and Roll Back

Version control is not meaningful if an organization cannot act when a problem appears.

Every consequential healthcare AI implementation should have a defined process for suspension, rollback, or isolation. Governance teams should know who can make that decision, what evidence triggers it, what version can be restored, and how affected users will be notified.

Incident review also depends on traceability. If a problematic output contributed to a patient-facing communication or clinical workflow, the organization should be able to reconstruct which system version produced it, what data or context were used, what the output was, and what human review occurred.

Without that record, post-incident analysis becomes guesswork.

Transparency Has to Survive the Update Cycle

ONC’s HTI-1 final rule established algorithm-transparency requirements for predictive decision-support interventions in certified health IT. That transparency is most useful when it stays current. Documentation that accurately described a model six months ago may be misleading after a material update.

Hospitals and health systems therefore need a simple rule: every meaningful AI change should force a question about whether prior documentation, validation, disclosures, training, and monitoring still describe the system that is actually in use.

That is the core discipline of AI version control.

Validation tells you that a particular system met a standard at a particular moment. Version control tells you whether that is still the system affecting care today. In healthcare, where software changes can influence clinical judgment, patient communication, workflow priority, and organizational accountability, that distinction is not administrative detail. It is part of safe AI operations.

About Dr. Michael T. Neely

Dr. Michael T. Neely is an independent researcher and AI educator whose work focuses on responsible AI, governance, accountability, patient protection, and human oversight. He is CEO of Global Music Conference and XS10 Magazine and brings more than four decades of leadership in the creative industries. His healthcare-AI commentary has appeared in KevinMD and Healthcare Business Today. His work examines how organizations translate responsible-AI principles into operational controls.