OpenAI may have just proven why third-party evaluations are crucial to frontier labs.

OpenAI hosted two staff members from AI safety evaluation firm METR and Redwood Research, giving them company devices with access to OpenAI’s codebase and internal infrastructure, as well as the chance to interview technical staff, according to METR. The evaluators identified gaps and vulnerabilities, and OpenAI has since told METR that “all of the vulnerabilities identified that impact their production safeguards stack have been patched.”

The pilot took place for three weeks from July 21 to August 11, coincidentally the same day OpenAI disclosed the Hugging Face incident. METR discloses that it did not evaluate later changes, such as expanding monitoring to reinforcement learning training, but rather analyzed the monitoring as it existed at the start of the pilot. Some of the vulnerabilities identified included:

• A way for agents to conceal hundred of actions from monitoring;
• The monitoring system was only triggered when agents returned control to the user, meaning they could circumvent monitoring by continuing to take actions;
• Due to the coverage gap, METR estimates that "on the order of 10% of internal Codex traffic was unmonitored from May to July." It says OpenAI had also independently identified the issue shortly before the exercise began.

OpenAI has since said that the findings helped "strengthen safeguards" and will help shape future collaborations with METR and other third party evaluators. METR recently published a proposal for third-party teams to assess control and monitoring of internal agents, comprising five core subquestions. For the OpenAI pilot, the evaluators focused on two:

What fraction of agent traffic was monitored, and could agents circumvent this monitoring?

Our Deeper View

This highlights the exact reason that there is a movement pushing for third-party evaluations: they can find blind spots initially missed. Even leaving aside the incentive that companies may have to handle evaluations internally and leave out safety pitfalls, assuming all positive intent, at the rate and speed the companies are moving, it is possible that they let things slip through the cracks. Additionally, when you are too close to a subject, it's easier to miss red flags. It's the same reason that journalists have copy editors — not because we don't understand grammar or know how to write, but because we have to move quickly and can become too attached to stories to see gaps in them.