SaMD AI Governance: The Governing the Algorithm Series
An introduction and 12 articles on SaMD AI governance: what it takes to keep an AI medical device trusted after clearance. It covers data governance, model risk management, human oversight, change control, accountability, ISO 42001 and the convergence of FDA and EU AI Act expectations, and closes with the five-layer governance stack.
13 articles in this series, in reading order
-
Governing the Algorithm · Introduction
Why AI Governance Is the Next Compliance Frontier in Healthcare
Launch compliance is giving way to operational compliance. Why AI, which can drift and be retrained after clearance, needs governance in a way drugs and fixed devices never did.
-
Governing the Algorithm · Article 1 of 12
Why "It's FDA Cleared" Is Not the Same As "It's Governed"
A 510(k) assesses one version of a product, on one dataset, at one moment. Why being cleared says little about how an AI is governed after that moment.
-
Governing the Algorithm · Article 2 of 12
The Governance Gap: Why Most AI Vendors Have a Model, Not a Management System
Most AI vendors describe their model when asked about governance. A model is an asset; governance is the management system that decides how it learns, changes and is monitored.
-
Governing the Algorithm · Article 3 of 12
Who's Actually Accountable When Your AI Is Wrong?
When an AI-assisted decision goes wrong, accountability runs through vendor, hospital and clinician. Why that chain has to be mapped before an incident, not after one.
-
Governing the Algorithm · Article 4 of 12
Garbage In, Liability Out: Why Data Governance Is a Clinical Safety Issue
In healthcare AI, bad input data produces bad clinical output. Why data governance belongs with clinical safety rather than in an engineering back office.
-
Governing the Algorithm · Article 5 of 12
The Bias Nobody Tested For: Why Demographic Performance Gaps Are a Governance Failure
A strong overall accuracy figure can hide worse performance for specific patient groups. If subgroup performance was never tested, nobody knows the gap is not there.
-
Governing the Algorithm · Article 6 of 12
Model Drift Is Inevitable. Undetected Model Drift Is Negligence.
Every deployed model drifts as equipment, populations and protocols change. Drift is expected; failing to detect it is the governance failure.
-
Governing the Algorithm · Article 7 of 12
Human-in-the-Loop Is a Design Requirement, Not a Marketing Line
"Human-in-the-loop" only protects patients if the clinician can actually disagree. How interface defaults turn claimed oversight into automation bias, and what exercisable oversight requires.
-
Governing the Algorithm · Article 8 of 12
Why Your PCCP Might Matter More Than Your Original Clearance
A PCCP sets out in advance how a model may change and how each change is validated. For AI that keeps learning, that plan can matter more than the original clearance.
-
Governing the Algorithm · Article 9 of 12
Post-Market Surveillance for AI: What Hospitals Are Starting to Demand
A validation study is a snapshot. Hospitals increasingly expect ongoing performance reporting after go-live, and vendors need a post-market surveillance plan that delivers it.
-
Governing the Algorithm · Article 10 of 12
ISO 42001 and the Rise of AI Management Systems in Healthcare
ISO 42001 does for AI governance what ISO 13485 did for quality: gives buyers an auditable way to verify it. Why healthcare AI vendors will increasingly need it.
-
Governing the Algorithm · Article 11 of 12
The EU AI Act, FDA, and the Coming Governance Convergence
The EU AI Act and the FDA reached similar governance expectations independently. What their convergence means for SaMD companies that sell on both sides of the Atlantic.
-
Governing the Algorithm · Article 12 of 12
The Governance Stack: Five Layers Every SaMD AI Company Needs Before Scale
The series closes on five layers every SaMD AI company needs before scaling: data governance, model risk management, human oversight, change control and accountability.