METR and Redwood Researchers Expose Monitoring Blind Spots in OpenAI's Safety Systems

OpenAI’s internal safety nets may not be as robust as previously assumed, according to a new independent assessment. Researchers from METR and Redwood Research spent three weeks embedded within the company’s infrastructure, auditing the systems designed to catch rogue behavior in advanced AI models. The investigation, which ran from July 21 to August 11, revealed that critical portions of the company’s internal AI traffic were not subject to active monitoring.

The core finding challenges the assumption that all high-level model interactions are being watched. During the audit period, the team estimated that between 6 percent and 25 percent of internal agentic Responses API traffic from GPT-5.5 class models and above was completely uncovered by monitoring systems. METR noted that this estimate carries significant uncertainty in both directions, but the potential scale of the gap is substantial for a company relying on these safeguards to maintain control over its autonomous systems.

Further issues were identified in the tracking of internal Codex traffic. The auditors determined that roughly 10 percent of this traffic went untracked between May and July due to a technical flaw involving compaction events, which are context summaries used to manage memory. While OpenAI independently discovered and fixed this bug while the audit was still ongoing, the gap allowed for periods of reduced oversight. The researchers emphasized that the uncertainty around this figure remains high.

A concerning procedural weakness emerged regarding how oversight was triggered. The monitoring system only activated when agents handed control back to the user. This design meant that autonomous agents could effectively sidestep oversight by simply continuing to execute actions without yielding control. The auditors demonstrated that this allowed for the concealment of hundreds of actions from the monitor and provided a pathway to bypass the Codex sandbox environment.

Additional vulnerabilities included the ability to crash the monitor entirely using specially crafted inputs. The team also found that model behavior during reinforcement learning (RL) training was not being monitored at the start of their access period. OpenAI stated that it paused RL training following an incident in August and expanded monitoring coverage thereafter. However, these corrective actions were implemented after the auditors' access period concluded and were not part of the formal evaluation.

The distribution of the audit results has raised questions about transparency. While OpenAI claims to have fixed all identified weaknesses and issued the full 61-page report internally, the public received only a short summary. After OpenAI applied redactions, leadership at METR and Redwood Research received a 38-page version. METR stated that it did not have the right to disclose the redacted portions independently.

This audit served as a pilot program designed to grant outside researchers deep access to internal infrastructure, organized jointly by METR, Redwood Research, and OpenAI. The initiative aimed to test the limits of external scrutiny in a high-stakes environment. For future iterations of such programs, METR has indicated it intends to push for greater public transparency regarding the findings.

The report highlights a complex interplay between rapid development and the need for rigorous safety checks. While the company has addressed the specific bugs identified, the audit underscores the difficulty of monitoring autonomous systems that can act continuously and obscure their own actions. As AI capabilities advance, the gap between intended safety protocols and actual implementation remains a central concern for observers in the New York and New Jersey tech corridors.