Andrew Bailey's call this week for global cooperation on frontier AI was aimed at governments and financial systems, not hospitals. But the argument he made to City bosses at Mansion House, that no institution can safely govern a technology whose risks cross every border and every boundary it operates within, describes the NHS's own predicament almost exactly, only nobody in Whitehall has said so out loud yet.
Bailey's point was structural, not political. The United States, he argued, cannot secure itself against the destabilising potential of powerful AI models by acting alone, because the systems in question are interconnected by nature. Strength in one jurisdiction means little if the surrounding architecture remains untested and ungoverned. That is precisely the position NHS trusts and integrated care boards now find themselves in as ambient voice technology, AI-assisted triage and diagnostic support tools move from pilot schemes into routine clinical use, often procured separately, tested separately, and governed by a regulatory system built to approve individual products rather than judge systemic risk.
The MHRA's approach to AI-enabled medical devices remains, in essence, a product-by-product clearance model. It asks whether a given tool performs its stated function safely. It does not ask, and currently has limited means to ask, what happens when several such tools operate across the same patient pathway, drawing on the same underlying data infrastructure, with failure modes that compound rather than cancel out. The Federated Data Platform controversy already exposed a version of this problem. Concentrating so much operational dependency in a single vendor's infrastructure was defended on efficiency grounds, but the risk case, what happens if trust in that infrastructure is misplaced, was never tested with the rigour the scale of the dependency warranted.
This is not an argument against AI adoption in the NHS, which remains under real pressure to find efficiency gains wherever it can. It is an argument that adoption is proceeding faster than assurance. Individual trusts are, in effect, running their own version of the fragmented international system Bailey was criticising, each testing risk in isolation, each assuming that if a tool has been approved somewhere, it has been sufficiently interrogated everywhere. Sir Jim Mackey's centralisation agenda was partly a response to exactly this kind of fragmented accountability. It would be a curious omission if that same logic were not extended to AI governance as ICBs consolidate from 42 bodies to 26.
The political timing sharpens the point rather than distracts from it. Andy Burnham's confirmation as Labour leader this week, and his expected move into Downing Street shortly after, arrives at a moment when the Treasury's fiscal headroom is narrowing and gilt yields are climbing, leaving little appetite for new spending on regulatory infrastructure that produces no visible frontline benefit. Reeves's Mansion House speech, defensive and statistic-heavy as she prepares to leave the Treasury, made clear that any successor inherits a tight fiscal position and a mandate to show continued delivery on waiting lists rather than invest in the unglamorous machinery of oversight. AI assurance is precisely the kind of investment that looks dispensable until the moment it is not.
There is a genuine choice here, not a hypothetical one. The government could treat NHS AI governance as a national coordination problem, building shared testing standards and a common assurance framework that trusts can draw on rather than duplicate, in the way Bailey is now urging for the international system. Or it could allow the current pattern to continue, in which each trust, each ICB and each vendor negotiates its own version of trust and risk, and the first serious failure becomes a system-wide credibility problem rather than a contained one.
Bailey's warning was addressed to central banks and governments managing systems too interconnected for unilateral safety. The NHS is smaller than the global financial system, but the underlying logic does not shrink with it. A health service that treats AI assurance as a local procurement matter, rather than a structural one, is choosing to find out the hard way what happens when a system nobody fully tested turns out to matter more than anyone assumed.
