A pathology laboratory in south-east London falls quiet. Blood tests cannot be matched to patients, operations are postponed, and clinicians return to paper request forms that many younger staff have never handled. That was the position in June 2024, when a ransomware attack on a pathology supplier disrupted hospitals across the capital and took months to unwind. The episode is worth recalling alongside the latest intervention from Brad Smith, the president of Microsoft, who has argued that household appliances face tougher safety scrutiny than artificial intelligence.

Smith describes AI as both tool and weapon. He wants binding international rules that include independent testing before release and mandatory disclosure when autonomous agents are compromised. He regards voluntary pledges, such as those signed by technology executives at the White House, as a weak instrument, since they restrain the responsible and leave the rest untouched. He also names Britain as better placed to lead than either Washington or Brussels, pointing to the access the AI Safety Institute retains to new frontier models despite pressure from the Trump administration to restrict it.

The claim deserves scrutiny from health leaders, because the NHS is where it will be tested. The Institute evaluates frontier models for risks to national security, including cyber capability. A hospital trust never buys a frontier model. It buys a scheduling tool or an ambient scribe built by a supplier on top of one, and the assurance chain between the laboratory and the ward is thin. The government's ten-year health plan expects AI to carry a growing share of administrative and clinical work, and trusts are under pressure to adopt quickly because waiting lists measured in millions leave little room for caution.

Smith's disclosure proposal is the most practically relevant. Current duties to report incidents were drafted with data theft in mind, and software that takes actions inside a workflow sits awkwardly within them. If an agent mis-routed referrals or altered appointment records, a trust might not know for days, and a supplier would have little obligation to say. Procurement contracts could close some of that gap now, yet individual trusts rarely have the commercial weight to demand incident disclosure and audit rights from large vendors.

The proposed kill switch raises a separate question. Halting an agent is useful only if the service can function afterwards. Downtime procedures are rehearsed unevenly across the NHS, estates and IT backlogs are well documented, and the digital teams that would manage such an event are small. The pathology disruption showed how quickly manual fallback degrades when staff have not practised it.

Britain's own position carries the weakness Smith identifies elsewhere. The Institute's access to frontier models rests largely on voluntary agreement with developers, which is exactly the arrangement he considers inadequate. Responsibility for health AI is spread across the medicines and devices regulator, the care regulator, the information commissioner and a department that is absorbing NHS England. Accountability lines are being redrawn at the moment deployment accelerates. International leadership would require domestic statute and a clear answer to who is liable when a clinical system fails.

For executives, the immediate task is contractual and operational: insist on disclosure terms and test the fallback. For ministers, it is to connect the Institute's findings to health regulators and to legislate. Health-tech firms should expect assurance demands to harden. Patients will judge the whole effort by whether a failure is contained.

Britain may well be the right place to write global rules for AI. The first serious test of that claim will probably arrive in a hospital IT department, long before it reaches a summit communiqué, and the health service will be judged on how it holds.

‍