Fable 5's median thinking per call fell 50% in August, an audit finds
A developer spent six weeks logging everything Claude Code sent and received. The result is the most detailed measurement yet of what frontier models actually deliver to paying users — and it does not match the benchmarks.
Fable 5's median invocation carried 123 thinking tokens, and 39.2 percent of invocations carried none at all, according to a six-week forensic audit published by developer Lon Lundgren under the title "The Inference Gap." The corpus is not a toy: 43,261 discrete Fable 5 invocations with quantified thinking totals, 7,853 complete assistant turns, 213 sessions, 25 project groups, three machines and two subscription accounts, collected from July 1 through September 7 across 48 Claude Code versions. Every record in the main analysis ran at xhigh or max effort — the settings Anthropic's own documentation describes as "always thinks deeply" and "always thinks," with no constraint on thinking depth.
The month-over-month comparison survived every reweighting Lundgren threw at it. Median turn thinking fell from 2,301 tokens in July to 1,797 in August, a 21.9 percent decline; weighting each project equally rather than each turn pushes the drop to 46.2 percent, and at the invocation level the median halved, from 156 tokens to 77. Around August 22 and 23, median turn thinking sat roughly 41 percent below its July reference while median invocation thinking hit zero for about a week. A partial recovery came near September 1, and then it turned down again.
The benchmark gap is the part that should worry buyers. Reconstructing the inference regimes behind published ARC-AGI-2 and HLE results, Lundgren puts Fable 5's blended thinking at about 24,400 tokens per task at xhigh and 32,100 at max. Against a delivered median of 123 tokens per invocation, that is eight token-doublings — a 230-fold shortfall, and roughly 38 times short even at the 75th percentile. Benchmark scores rise non-linearly with thinking compute, so the practical penalty is far larger than the missing percentage suggests. Thinking also fragments as turns grow longer: past a few thousand thinking tokens, the longest uninterrupted reasoning span stops dominating the turn, and in turns that reached 48 invocations, every one contained zero-thinking calls.
Lundgren is explicit that the data does not identify a cause — capacity management, routing, adaptive effort logic or a policy change all fit — and Anthropic has not responded to the analysis. But the mechanism matters less than the measurement, because Anthropic already routes Fable 5 requests to other models server-side when safeguards trigger, and effort levels are documented as a behavioral signal rather than a token budget. A customer cannot tell a provider-side change from their own bad prompting, and that is the actual defect. We caught the same failure mode in miniature in August — Claude Code quietly turned 'high' effort into what 'low' used to be — one silent effort remapping, found by one developer refusing to believe his setup was broken.
What to watch: whether any lab starts publishing realized reasoning tokens per call, the share of invocations with zero thinking, and how those distributions move over time. Those four numbers would settle every "the model got dumber" argument in the industry.
If your agent felt worse last month, would a provider-side metric have proven it — and what would you want published? Tell us in the comments.
Sources: Lon Lundgren — The Inference Gap · Lon Lundgren on X — median thinking declined in August · Hacker News discussion · Anthropic — change the model, effort, and thinking settings · Anthropic — why Claude switched models in your conversation
