Some of AI's most prominent figures are urging the industry to rein in the tech's most pressing risks.

On Tuesday, a coalition of more than 400 researchers, including AI godfathers Yoshua Bengio and Geoffrey Hinton, published an open letter calling for AI and security researchers to collaborate on improving alignment and containment approaches, and aiming to protect critical infrastructure before more incidents of "high-profile security failures" come to fruition.

The letter claims that these incidents both serve as evidence of the importance of AI alignment and could have been preventable with stronger sandboxes, representing an "irresponsible lapse in standard cybersecurity practices." Though it didn't name any one company specifically, the letter comes in the wake of OpenAI's July breach of Hugging Face and the waterfall of incidents that have come to light since involving several major AI labs.

"Without immediate changes in AI development and deployment practices, such incidents will continue to occur, and future incidents will be far more damaging to our exposed critical infrastructure," the letter reads.

The letter makes three recommendations for addressing these risks:

• Organizations that are developing and deploying "cyber-capable" AI models should immediately implement "high-assurance containment systems," as well as initially default to "airgapping" these models, or isolating them from unsecured external networks.
• It suggests the creation of an international standards body that includes governments and a wide range of AI and security experts to create best practices for alignment and containment that "allow for greater access outside the sandbox." This organization should also track all containment failures, the letter notes.
• Finally, all security incidents involving AI containment failures must be reported to this standards body and made public, allowing the industry to learn from its mistakes. "This transparency is required for both learning and accountability purposes," the letter said.

This letter adds to a growing number of AI industry researchers and leaders calling for increased safety guardrails. And the calls are coming from inside the Big Tech house: Microsoft's Satya Nadella, former DeepMind CEO Demis Hassabis, and Anthropic's Dario Amodei have been among the industry leaders in recent months urging for stronger security standards.

However, despite the calls for transparency, AI labs might be getting fickle on what they tell the public: A report from Reuters published on Tuesday claims that Anthropic and OpenAI are seeking influence over what these evaluators can publish. Though the report notes that this aims to prevent intellectual property exposure, this could also limit the transparency and utility of third-party evaluations.

Our Deeper View

Though many in the industry have voiced their concern about humans losing control of AI, this letter argues that many of the incidents of rogue AI thus far have been a result of weak sandboxes, and could have been prevented with stronger security measures from the jump. However, these calls for containment come amid increased discussion around the possibility of recursive self-improvement, or RSI, as labs race toward AI that can improve upon itself. Jack Clark, a cofounder of Anthropic, wrote in a Substack newsletter on Monday "full RSI" will become possible in 2027, rather than 2028 like he had previously predicted. If these models' self-improving capabilities are truly around the corner, then strengthening our ability to control them before we get there is more important than ever.