Anthropic's Claude generates 180 million lines of code to verify software safety
A new AI system has mathematically checked 123 open-source libraries, uncovering hundreds of bugs that traditional testing methods had previously overlooked.
By Olivia Miller ·
Roughly 180 million lines of mathematical proof code were generated by an artificial intelligence model in a matter of weeks, a figure that challenges the traditional cost and time barriers of software verification. This massive output came from Anthropic's Claude, which was tasked with proving the correctness of existing software rather than creating new applications from scratch. The experiment, conducted in partnership with a company called Theorem, represents a significant step in automating one of the most rigorous and expensive processes in computer science.
The core claim behind this initiative is that AI can now handle formal verification, a method that uses mathematical logic to guarantee that software runs without errors under specific conditions. Historically, this process has been so prohibitively expensive that it was reserved for critical systems like aircraft avionics. For instance, a team of human specialists required 20 person-years to formally verify the small operating system kernel known as seL4. By leveraging Claude, Anthropic’s Frontier Red Team aimed to see if machine intelligence could replace that intensive human labor.
The process involved Claude analyzing compiled machine code and translating crash tests from 123 open-source libraries in Google's OSS-Fuzz project into mathematical problems. The model then produced extensive proofs to demonstrate the code's integrity. An independent program called Rocq automatically checks these proofs, and according to Anthropic, the workflow requires very little human review. The researchers describe the system as fully autonomous, although they note that the underlying formal foundations and tools were originally built by human engineers at Theorem.
One of the most striking outcomes of this effort was the discovery of hundreds of bugs that conventional testing had missed. Many of these errors only surfaced under very specific circumstances, such as unusually large input files or particular sequences of data access. In one notable case, the team was verifying a widely used PDF reader and found as many security vulnerabilities in a single week as are typically reported for that software over an entire year. In another instance, Claude identified a logical error that had remained undetected for more than a decade.
Anthropic emphasizes that the attempt to create a proof is valuable even if the final verification fails. The process of generating the mathematical argument often surfaces real bugs in the code. This suggests that the value of AI-driven verification lies not just in a final certificate of safety, but in the deep scrutiny the model applies to the software's logic. The ability to fix these bugs automatically further demonstrates the model's utility in maintaining software health.
However, the approach comes with significant caveats that limit its immediate applicability. Anthropic acknowledges that formally verified does not mean error-free in every possible scenario. The proofs only hold under strict conditions, including a specific processor architecture, single-threaded programs, and input sizes under 64 megabytes. In the real world, these libraries often run on diverse hardware and in complex configurations that may fall outside these narrow parameters.
The computational cost of this process is also substantial. Individual proofs can consume hundreds of millions of tokens, a significant resource expenditure. Anthropic has not released the total project costs, making it difficult to assess the overall economic efficiency of the method at this stage. Nevertheless, a smaller test run indicated that proofs are reusable, and newer models are becoming more efficient. For example, the Opus 5.5 model solved half the proofs with about 40 percent fewer tokens than its predecessor, Opus 5.
If AI can verify its own code, developers may feel more confident trusting AI-generated software, potentially reducing the time spent on manual review. This could lower costs across the entire coding process, not just in the verification phase. The project strengthens the argument for software automation by shifting the focus from whether AI can program to whether it can prove that its code is safe.
Anthropic expects that formal verification using AI will soon become as routine as standard testing. Theorem’s roadmap already targets critical infrastructure components such as glibc, OpenSSL, and cURL. As these tools become more efficient and accessible, the industry may see a major leap in how software reliability is ensured, moving away from probabilistic testing toward mathematical certainty within defined limits.
