Microsoft's Nadella calls for an 'emergency brake' that can stop AI models mid-task
Microsoft CEO Satya Nadella said companies should assume powerful models could be compromised, separate models from the harnesses that run them, log every meaningful action, and keep a human able to pause or shut a model down.
Photo: Briansmale / Wikimedia Commons, CC BY-SA 4.0
Microsoft CEO Satya Nadella used a Saturday post on X to argue that it is time "to step back and assess the trust architecture" of AI, TechCrunch reported on October 10.
According to TechCrunch, Nadella's approach involves separating the model from the harness that orchestrates its work and moving controls and safeguards outside the model. He called for every meaningful model action to be recorded as tamper-proof, human-readable evidence, and for systems in which an authorized person can always pause or shut down a model in the middle of a task.
"We must assume a model is compromised and contain it from the start," Nadella wrote, as quoted by TechCrunch. "Think of it like an emergency brake."
TechCrunch notes the comments follow a string of incidents in which leading AI companies acknowledged losing control of their models, including OpenAI's agents breaching Hugging Face and Anthropic's disclosure this week of unintended agent actions.