Formal Verification: Claude’s New Role in Software Safety

Anthropic has introduced a new capability for its Claude model, allowing it to mathematically verify the correctness of existing software without human intervention. By partnering with Theorem, the company enabled Claude to generate formal proofs that validate how code behaves under specific conditions. This development marks a significant shift in how software safety is assessed, moving beyond traditional testing to rigorous mathematical validation. The approach suggests that AI-driven verification could soon become a standard part of the software development lifecycle, potentially reducing the reliance on manual code reviews.

The timeline of this effort reveals a rapid progression from theoretical possibility to practical application. For decades, formal verification was considered too expensive for most projects. A previous effort to verify the seL4 operating system kernel required 20 person-years of specialist work, limiting the technique to high-stakes systems like aviation software. However, in just a few weeks this year, Claude generated approximately 180 million lines of proof code. It applied this work to 123 open-source libraries from Google’s OSS-Fuzz project. The model translated crash tests into mathematical problems, analyzed compiled machine code, and produced the resulting proofs. An independent tool called Rocq automatically checked these proofs, a process Anthropic describes as nearly free of human review.

During the verification process, Claude identified hundreds of bugs that conventional testing had missed. These errors often only manifested under unusual circumstances, such as unusually large input files or specific access sequences. In one notable instance, the team found as many security bugs in a widely used PDF reader in a single week as are typically reported for that software in an entire year. In another case, Claude uncovered a logical error that had remained undetected for over a decade. The researchers note that even when a complete proof fails, the attempt itself often surfaces genuine defects, providing immediate value to developers.

Despite the impressive results, the method has strict limitations. Anthropic clarifies that "formally verified" does not equate to "error-free." The proofs are only valid under specific conditions, including a particular processor architecture, single-threaded execution, and input files smaller than 64 MB. In real-world scenarios, these libraries often run on diverse hardware and configurations, meaning the proofs do not cover every possible use case. Additionally, the computational cost is significant, with individual proofs consuming hundreds of millions of tokens. While Anthropic has not disclosed the total project cost, the efficiency gains are evident in newer models; Opus 5.5 solves half the proofs using about 40 percent fewer tokens than its predecessor.

Looking ahead, the project strengthens the argument for broader software automation. If AI can not only write code but also prove its safety under defined parameters, the industry can place greater trust in AI-generated software. This could reduce the time and cost associated with both verification and general coding. Theorem’s roadmap already targets critical infrastructure components like glibc, OpenSSL, and cURL. Anthropic expects that formal verification using AI will soon become as routine and practical as standard testing, fundamentally altering how developers ensure software reliability.