While many questions around AI safety and alignment remain unanswered, Microsoft's CEO is calling out a different challenge: trust.

On Saturday, Microsoft's Satya Nadella published an essay arguing that models present "insider risks," and that these systems' access, observability and authority pose a bigger threat than the capabilities themselves. His primary argument is that these models should be treated with zero trust principles: As these systems are non-deterministic, they should be handled with the same security guardrails as a powerful employee, and assume that they can make mistakes or be compromised.

Nadella sets aside the conversation around alignment, instead suggesting that an "engineering approach to containment and governance" is a vital step in controlling and understanding AI.

"We can’t treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions," Nadella wrote in the essay published on X. "We must build contained systems whose behavior we can observe, limits we can test, and actions we can always contain."

Nadella suggested several standards of observability that should be built into enterprise AI, noting that this should sit external to its orchestrator and harness so that the model cannot access or change them.

• No single model should be used for all important outcomes, or be responsible for checking its own work.
• Every meaningful AI action should leave "tamper-proof human readable evidence," allowing administrators to reproduce how models came to their conclusions.
• AI should be continuously tested and verified, including failures, attacks and edge cases.
• Model validation can't be completed by the model that's being validated. Additionally, organizations should be able to independently control the model.
• Assume that a model has been compromised and contain it from the jump by building an "emergency brake."
• And finally, if these systems do fail, those failures should be disclosed in a timely manner.

"We simply can't outsource responsibility for what intelligence does on our behalf," Nadella wrote. "A model provider’s assurances do not relieve us of that responsibility."

Nadella's essay isn't the first time a Microsoft leader has called for a more measured approach to AI safety. Last month, Microsoft AI head Mustafa Suleyman wrote an essay warning of the risks of treating AI as if it were conscious, noting that training it as such could allow it to circumvent safety guardrails, as well as condition its creators and users to believe that it deserves rights as a "moral patient."

Our Deeper View

Nadella and Suleyman's essays both draw on a similar theme: that AI may be a powerful tool, but it is not a being that should be trusted to keep itself in check and take mission-critical actions on behalf of an organization. Both essays also underscore Microsoft's measured, level-headed approach to AI that sets it apart from the major model labs. Though Microsoft isn't exactly thriving in putting out powerful, frontier models compared to its competitors, it doesn't need to compete in that way. Its strategy isn't to create powerful intelligence, but rather to pinpoint and target the most pressing problems that enterprises are facing. For instance, its model releases primarily center niche use cases and token efficiency. Its governance stances, meanwhile, center trust and security. And given the company's large existing market share in workplace and productivity technology, it shows that Microsoft is a company that understands its audience very well.