BIG TECH

Microsoft reframes AI safety debate around trust

While many questions around AI safety and alignment remain unanswered, Microsoft's CEO is calling out a different challenge: trust. 

On Saturday, Microsoft's Satya Nadella published an essay arguing that models present "insider risks," and that these systems' access, observability and authority pose a bigger threat than the capabilities themselves. His primary argument is that these models should be treated with zero trust principles: As these systems are non-deterministic, they should be handled with the same security guardrails as a powerful employee, and assume that they can make mistakes or be compromised.  

Nadella sets aside the conversation around alignment, instead suggesting that an "engineering approach to containment and governance" is a vital step in controlling and understanding AI. 

"We can’t treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions," Nadella wrote in the essay published on X. "We must build contained systems whose behavior we can observe, limits we can test, and actions we can always contain." 

Nadella suggested several standards of observability that should be built into enterprise AI, noting that this should sit external to its orchestrator and harness so that the model cannot access or change them. 

• No single model should be used for all important outcomes, or be responsible for checking its own work.

• Every meaningful AI action should leave "tamper-proof human readable evidence," allowing administrators to reproduce how models came to their conclusions.

• AI should be continuously tested and verified, including failures, attacks and edge cases.

• Model validation can't be completed by the model that's being validated. Additionally, organizations should be able to independently control the model.

• Assume that a model has been compromised and contain it from the jump by building an "emergency brake." 

• And finally, if these systems do fail, those failures should be disclosed in a timely manner. 

"We simply can't outsource responsibility for what intelligence does on our behalf," Nadella wrote. "A model provider’s assurances do not relieve us of that responsibility." 

Nadella's essay isn't the first time a Microsoft leader has called for a more measured approach to AI safety. Last month, Microsoft AI head Mustafa Suleyman wrote an essay warning of the risks of treating AI as if it were conscious, noting that training it as such could allow it to circumvent safety guardrails, as well as condition its creators and users to believe that it deserves rights as a "moral patient."

Nadella and Suleyman's essays both draw on a similar theme: that AI may be a powerful tool, but it is not a being that should be trusted to keep itself in check and take mission-critical actions on behalf of an organization. Both essays also underscore Microsoft's measured, level-headed approach to AI that sets it apart from the major model labs. Though Microsoft isn't exactly thriving in putting out powerful, frontier models compared to its competitors, it doesn't need to compete in that way. Its strategy isn't to create powerful intelligence, but rather to pinpoint and target the most pressing problems that enterprises are facing. For instance, its model releases primarily center niche use cases and token efficiency. Its governance stances, meanwhile, center trust and security. And given the company's large existing market share in workplace and productivity technology, it shows that Microsoft is a company that understands its audience very well. 

TOGETHER WITH OPTIMIZELY

Stop asking what AI should do

Most AI planning starts with the wrong question. Teams ask "what should AI do?" when the better question is "where does the work actually break?"

Optimizely's AI Use Case Discovery Template flips that script and gets you to a real answer fast.

• No blank page: Answer a few questions about your workflow and get a personalized AI use case. No worksheet required.

• Proven process: It's built on the same approach that turned a 2-3 hour GA4 reporting task into a 5-minute one.

• Tailored to your team: You walk away with a use case mapped to how your team actually works, not a generic list of ideas.

• Minutes, not meetings: No framework deck, no facilitator needed.

Find your first AI use case →

OPEN SOURCE

The hidden risk in China’s open model boom

Conversations about model safety have swept US frontier labs. It's a different story in China. 

On Thursday, research firm SemiAnalysis published a report analyzing hundreds of model releases from nine leading Chinese labs over the past five years, finding that only a small percentage of them publish safety reports or disclosures alongside their releases. The analysis included four of the country's biggest hyperscalers, ByteDance, Alibaba, Tencent and Baidu, as well as five of its hottest startups, DeepSeek, MoonShot, Zhipu Z.ai, MiniMax, and StepFun. 

The firm analyzed 857 model releases between September 2021 and September 2026, including 741 product models and 116 research models. Of that pool, only 31 models, or around 3.6%, had safety reports attached to them, and only 9 of them were available at or before release. 

• Though Alibaba shipped the most models out of the group at 238, only seven of those were accompanied by safety reports. Tencent, Baidu and ByteDance had only four safety reports for the dozens of models they released. 

• As for the startups, only Zhipu Z.ai has shipped a safety report every year since 2022, but still has published only nine safety reports of the 131 models it's released. It's also the only lab in the group with a strategic commitment to safety, according to SemiAnalysis. 

• Deepseek has published six safety reports for its 93 releases, StepFun has three for its 39, and Moonshot and MiniMax have one each for their respective 34 and 20 releases. 

It should be noted that the SemiAnalysis report didn't include analysis of US labs' safety reports upon release. However, amid a number of recent incidents involving agents breaking through sandboxes, safety conversations among these labs have intensified:  The leaders of two of AI's biggest power players, Anthropic and OpenAI, have both stated their concerns about safety outpacing development, with both companies in recent months promising to slow development of their most powerful models, though those same labs have released a number of new models in recent weeks. 

However, these safety conversations also come at a time when enterprises are increasingly searching for alternatives to expensive proprietary APIs as they clamp down on excess AI spend and search for returns. While neolabs like Jev and Pathway are starting to gain traction as a result, many are turning towards open-source models, the bulk of which come from China, for cost and efficiency.

The prisoner's dilemma of China versus the US has often been used as a justification against pacing frontier model development, both by major model labs and government officials. However, Chinese labs have also been accused on several occasions of excessive distillation to siphon off the capabilities of the most powerful models from their US counterparts. Though a large-scale slowdown of AI development would require a coordinated geopolitical effort, the labs putting safety at the forefront and slowing down could have a positive impact on the broader ecosystem, with ripple effects that could impact Chinese labs. However, as it stands, because a growing number of enterprises are turning to open-source Chinese models to save money and have more control, the enterprises relying on these models need to be aware that the onus for safety falls on them. 

TOGETHER WITH NOOKS

Sales is harder than code. Nooks Labs is working on it.

Coding agents get feedback in seconds: the tests pass or they fail. Nooks learned that sales agents can wait weeks to find out whether a move worked. Much of what matters, like who's really making the decision, never makes it into the CRM.

Nooks' agents scored 97% on the company's own eval set, but reps still double-checked every output. So Nooks Labs, the company's applied AI team, built a harder benchmark from real deal data. The team then fixed what it exposed with calibrated confidence scores, a way to test a strategy against past deals before it runs, and 500+ examples of bad AI writing used as guardrails.

Today, 96% of agent-written emails go out without edits. The open question now: what does it take to go from 2x productivity gains to 10x?

Discover the research →

GOVERNANCE

Why AI agents need a flight recorder

Companies want to own the intelligence that makes their business valuable. So what will that mean for the AI models they use, and the systems they build around them?

In this episode of The Deep View Conversations, we sit down with Justin Boitano, VP of enterprise AI at Nvidia, to explore why NVIDIA is investing in open models and how enterprises are turning them into specialized intelligence.

Boitano talks about why NVIDIA makes Nemotron's datasets, training techniques and model weights openly available, how proprietary business knowledge can become a competitive advantage, and why lower token costs can unlock much greater demand for AI.

The conversation also examines why the agent harness that manages tools, memory and execution can matter as much as the model itself. Boitano shares how Nvidia worked with CrowdStrike on AI models for cybersecurity, why AI incidents need the equivalent of an aviation flight recorder, and how OpenShell moves agent controls into the infrastructure layer.

Other topics covered:

• Why NVIDIA is building open models alongside its chip business
• What Nemotron needs to compete with China's open models
• Owning institutional knowledge, protecting data and controlling AI costs
• How model development helps NVIDIA improve hardware efficiency
• Why harness engineering is becoming a new frontier for AI
• Moving cybersecurity from finding bugs to verifying and fixing them
• Transparency and learning from AI incidents
• OpenShell, sandboxing and enforceable limits on agent access
• How Justin uses Hermes as an AI chief of staff
• Jensen Huang's approach to priorities and team alignment

If you're deciding how to build enterprise AI, where your company's competitive advantage comes from, or how to give agents useful access without giving them unlimited control, this conversation offers a perspective from Nvidia's work across the ecosystem.

📺 Watch on YouTube
🎧 Listen in your favorite podcast player

Subscribe to Deep View Conversations for interviews with the leaders shaping the future of AI, business, and technology: https://tdv.transistor.fm/

LINKS

• Anthropic agents submitted a false murder tip to Philadelphia police 

• Three AI researchers say OpenAI fired them for prioritizing safety

• Meta launched Muse despite safety concerns after Instinct gained traction

• Altman-backed Meanwhile raised $37.5 million at $350 million valuation

• Nvidia, Oracle, CoreWeave shares fell following OpenAI revenue details

• TypeSafe AI raised $870 million led by a16z as Jev tops one million users

• Microsoft-Decision-1: Microsoft's fastest decision-making model yet

• Firefly Video Editor: Adobe connects Claude Motion with Firefly Video Editor

• Arena Alignment Index: new benchmark measuring safety of AI agents in real-world use

• Grok Bot: can now search, read, and monitor X

• Ultrafast: is rolling out for GPT-6.1 Sol in the API, Codex, and ChatGPT Work

• Clarity Recruiting: Junior AI Specialist

• Fanatics: AI Engineer

• Liquid AI: Product Manager

• OpenAI: Agent Post-Training, API & Power Users

GAMES

A QUICK POLL BEFORE YOU GO