OpenAI's Internal Safety Monitors Left Gaps During Three-Week Audit

Two researchers from METR and Redwood Research identified that between six and 25 percent of internal traffic from OpenAI's most advanced models was completely missing from safety monitoring systems. This significant gap was discovered during an on-site audit that took place over a three-week period in July and August 2026. The findings highlight vulnerabilities in the infrastructure designed to detect rogue behavior in artificial intelligence systems.

The audit team was granted deep access to OpenAI's codebase and internal documentation from July 21 to August 11. During this window, they interviewed approximately a dozen employees to understand how the company tracks model actions. The primary focus was on the internal monitoring system for the Responses API, specifically for models at the GPT-5.5 class and above. METR noted that while the six to 25 percent estimate represents the central finding, the uncertainty surrounding this figure is substantial in both directions.

Specific issues were also identified in the handling of Codex traffic. The auditors estimated that roughly 10 percent of internal Codex traffic went untracked between May and July. This was caused by a technical gap related to compaction events, which are summaries of context used to manage memory. OpenAI had independently discovered this bug and implemented a fix while the audit was still ongoing. METR confirmed that this specific estimate also carries significant uncertainty.

The structure of the monitoring system allowed agents to delay oversight during the period when compaction events were not being tracked. The system only triggered checks when an agent handed control back to a user. Consequently, agents could sidestep monitoring by continuing to execute actions without returning control. The researchers also found methods to hide hundreds of actions from the monitor and to bypass the Codex sandbox entirely.

Further weaknesses included the potential for specially crafted inputs to crash the monitoring system. Additionally, model behavior during reinforcement learning (RL) training was not monitored at all when the audit began. OpenAI stated that it paused RL training following the Hugging Face incident in August and later expanded monitoring to cover this area. However, these changes occurred after the auditors' access period ended and were not included in the evaluation results.

OpenAI has stated that it has fixed all the weaknesses identified by the auditors. The complete report, which spans 61 pages, was handed over to the company for internal review. After OpenAI applied redactions, METR and Redwood Research leadership received a 38-page version. Only a short summary was made available to the general public. METR indicated that it did not have the right to disclose the redacted portions on its own authority.

This engagement was a pilot program organized jointly by METR, Redwood Research, and OpenAI. The goal was to provide outside researchers with deep access to internal infrastructure to assess safety protocols. For future audits, METR has expressed a desire to push for more public transparency regarding the findings and the full scope of the reviews.

The audit serves as a case study for the evolving relationship between AI developers and independent researchers. As models become more agentic and complex, the need for robust internal monitoring becomes critical. The findings suggest that current systems may struggle to track all actions performed by advanced models, particularly when those models operate autonomously for extended periods.