Claude Opus 5 lifts its test pass rate to 88.6% — and writes 2.3x more code

Code-analysis firm Sonar put Anthropic's Claude Opus 5 through its standard LLM evaluation and found the model passes more of the tests it's given — hitting 88.6% on executable coding tasks versus 82.9% for the Opus 4.8 it replaces — while generating more than twice as much code along the way.

The pass-rate gain is the headline. Across 544 HumanEval and MBPP tasks with executable tests, Opus 5 landed at 88.6%, a 5.7-point step up from Opus 4.8's 82.9%. The three Opus releases before it all clustered within a single percentage point (82.72%–83.66%), so this is a genuine jump rather than drift, and the model also finishes what it starts more reliably — missing completions fell from 1.22% to 0.83%.

On a per-line basis, the code got cleaner. Bug density dropped 14% and vulnerability density fell 20%, with Sonar reporting its most striking figure yet: blocker-level, security-impact findings tumbled 75%, from 77 to 19 per million lines of code. Exception-handling bugs fell 52% and control-flow mistakes 51%. Anthropic has framed Opus 5 as unusually good at verifying its own work while it plans; Sonar's numbers are broadly consistent with that claim.

The wrinkle is volume. For the same 4,441 tasks, Opus 5 produced 916,813 lines of code against Opus 4.8's 391,456 — a 2.3× jump — and 3.6× the output tokens (21.7 million versus 6.0 million). It split that into 2.5× as many functions, each slightly tighter, which Sonar reads as decomposition rather than sprawl. The catch is that cleaner-per-line still scales up: 2.3× more code translates to roughly 2.7× the total issues, with code-smell density up 17% and the collections-and-generics smell category more than doubling. Concurrency and threading bugs nearly doubled to become the top bug category — the same pattern Sonar saw in every GPT model it evaluated this month.

Both readings are true, and the practical takeaway is that verification load doesn't shrink — it shifts. Teams adopting Opus 5 should point automated checks where Sonar's data points: threading, cryptography configuration (its top vulnerability category, which jumped from 43 to 103 per million lines), and untyped collection handling. The volume tradeoff also complicates the model's core pitch as a cheaper alternative to Claude Fable 5 — more output means more code to review even when that output is denser with fixes. We saw the cost calculus already pulling buyers toward Opus 5 in Anthropic's Opus 5 overtakes flagship Fable 5 as buyers trade down.

What to watch: Whether Anthropic trims Opus 5's verbosity in a point release, and whether rival benchmarks start scoring code volume the way Sonar now does.

Which tradeoff matters more to your workflow — cleaner code or less of it? Tell us in the comments.

Sources: Claude Opus 5 code quality: what Sonar's benchmark reveals (HackerNoon) · Sonar LLM Leaderboard · Claude Opus 5 — Anthropic