Agents beat human accountants on the job — at 2.5x the cost
A new benchmark hires agents the way a company hires people — handbook, old records, a mentor — and the best of them now outperform the humans on accuracy. They also cost more.
Anthropic's Fable 5.1 finished a simulated accounts-payable job at 72% cumulative accuracy, against 51% for the best human tester — at $18.23 per invoice versus $7.21 for the human. NeoCognition built ApprenticeBench around a full year at a simulated California construction company, Acme Home Builders: the agent joins in May 2026, reads the company handbook, an Odoo tutorial and six months of posted bills, then processes 100 incoming invoices while a manager comments on every one for the first month and only at month end after that. OpenAI's GPT-6 Astra landed second at 68%, and the gap to the rest is the finding: Opus 5 reached 36%, Gemini 3.8 Flash 24%, and Kimi K3 — the best open-weight entry — 18%. Human time is priced at the U.S. Bureau of Labor Statistics' median wage for construction bookkeeping clerks, $26.39 an hour, so the cost comparison is wage-for-token, not apples-to-oranges.
The benchmark's designers call the numbers a step change, and the mechanics back it up. The "CUA tax" — the accuracy drop from using an app's graphical interface instead of its API — is gone for Fable 5.1 and GPT-6 Astra, and flipped negative for both, while older models lost as much as 78% of their accuracy working through the same screens a person uses. Continual learning is real but narrow: given only the handbook, Fable 5 solved 11 of 100 bills; adding history alone took it to 31, mentor feedback alone to 35, and both channels to 43. The strongest evidence of learning is on the 28 bills where no precedent exists and the cost-code convention has to be induced — Fable 5.1 and GPT-6 Astra got 22 of 28, the four weaker models between 7 and 10. Fable 5.1 needed 62.8 hours and 3.7 billion tokens to get through the year.
The honest reading is that this is a cost story before it is a capability story. Agents are now more accurate than the clerk and roughly 2.5x more expensive per task, and the expense grows with the run — humans get faster with practice while agents get slower, because their notes balloon. Fable 5.1 wrote 192,000 words across 122 memory files over the year; one human tester kept about 2,700 words in a single document and finished at 40%. Fable 5.1 also sent 1,032 messages against a designed minimum of 248, a 4.4x overhead ratio, where the humans ran about 2x. Today that arithmetic means a firm hires the agent only if it values accuracy over money — which is a real market, but a smaller one than the "independent workforce" framing implies.
Worth pairing with this month's other Fable 5.1 result — Fable 5.1 leads Real-SWE, and still fails 6 of 10 tasks — where the same model topped a private-code leaderboard at 38.8%.
Google is testing an unreleased math model under the internal tag deepthink-mathematica, with a reported 1-million-token context and a 65,536-token output cap. The build surfaced through API identifiers and was flagged UNSTABLE_EXPERIMENTAL, carrying Google's "Teamfood" internal-testing label — meaning it has no public endpoint, no published benchmarks, and no acknowledgment from Google, which has not commented on the leak. The reported lineage runs from Gemini Deep Think, which hit gold-medal standard at the 2025 International Mathematical Olympiad with 35 of 42 points, through Aletheia's research-level work, to this model, described as optimized for heavy computation and symbolic problem solving rather than conversation.
The detail drawing attention is the reasoning trace itself: the model reportedly emits exclamations mid-derivation — "wait", "oh my god", and the Chinese equivalents 等等 and 我的天. Treat the theatrical inner monologue as a symptom, not the story. The number that matters is the output limit: 65,536 tokens is a proof-length budget, several times a normal chat response, and it says Google is building for multi-step derivations that need to run to completion in one pass. Reported speed gains over the previous generation are unverified and single-source.
What to watch: whether NeoCognition rotates new job functions and company histories into ApprenticeBench, and whether any lab publishes a cost-per-task figure it is willing to defend next to a human wage.
If the best agent is 20 points more accurate than your clerk but costs 2.5x more per invoice, which one do you hire? Tell us in the comments.
Sources: ApprenticeBench — NeoCognition · ApprenticeBench leaderboard · BenchLM — ApprenticeBench scores · 36Kr — Google's Mathematica model leaks · Sohu — Mathematica spec sheet · Crypto Briefing — Google tests Deep Think Mathematica internally
