Plain LRU beat three smarter KV-cache policies on 68k real agent requests
An independent simulation just landed a null result on one of inference's busiest research corners. Plus: construction robotics is paying off — inside factories, not on job sites.
A reproduction of the fast-growing "LRU is wrong for agents" KV-cache literature could not beat plain LRU on any real trace. Cross-request prefix caching — reusing key-value blocks that were already computed so a coding agent's fiftieth turn costs a fraction of its first — is evicted with LRU by default in vLLM, SGLang, LMCache, and Mooncake. The counter-argument is intuitive: agent sessions go idle, and LRU can't tell a paused session from a dead one. The author believed it, built a discrete-event simulator, first validated it by reproducing Mooncake's published hit-rate-versus-capacity curve (the shape matched; he published an unexplained four-to-six-point offset rather than tune it away), then ran 68,266 requests from 393 real Claude Code sessions against three smarter policies. All three lost. A predictor of whether a session will return cost 1% to 4.5% more recompute than LRU; adding a realistic recompute-cost model pushed that to 15% to 90% worse; session-coherent eviction — the piece he was most confident in — was the worst at up to 208% worse. LFU collapsed to −447%.
The reason is the finding worth stealing. Measured policy-independently, a third of all recomputation (33.1%) comes from requests arriving less than 10 seconds after the last one — tight tool loops whose roughly 88,000-token working sets simply don't fit — versus 17.5% from gaps longer than five minutes. At a 2.1-second median gap between requests, a liveness predictor has nothing to discriminate on, and a provider-style five-minute TTL never fired once, producing byte-identical results to plain LRU because eviction always beat the timer. The regime is capacity-bound, not TTL-bound, and that changes the lever: compression, tiering, and admission control instead of cleverer eviction. Positron bets $875 million that inference doesn't need expensive memory attacks the same economics from the hardware side.
The repo's most useful paragraph is a harness warning: in the author's first run, Belady — an offline oracle with perfect future knowledge — lost to LRU. That isn't a result, it's a broken simulator: the eviction logic let the cache evict the very chain it was mid-way through building, a bug every non-recency policy cannibalises itself on unless in-flight blocks are pinned. His proposed smoke test for any KV-cache simulation is "does Belady beat LRU?" — it must. He also notes several published alternatives benchmark against degraded ports of their competitors, including two papers that tested Continuum with its adaptive TTL replaced by a fixed 2-second pin. Those margins read softer than published.
CNBC surveys construction robotics and finds the measurable wins are indoors, not humanoid. The U.S. is short roughly 1.2 million homes with nearly 300,000 construction jobs still open at the end of 2025, a labor gap NAHB chief economist Robert Dietz prices at about $11 billion a year and two added months per build — against a sector whose residential productivity has risen only 16% since 1993 versus more than 50% for the economy overall. The results that are real are narrow: Dusty Robotics' floor-printing layout machine covers 10,000 to 15,000 square feet a day (Skanska reported a 75% rework cut using it), while Japanese factories where robots build about 80% of a house lifted a Sekisui Heim line from 55 to 65 housing units a day with 20 fewer operators, on a roughly three-year payback. Dietz's verdict on national productivity impact: "It's too early." The honest read matches Dusty's founder — the binding problem is information loss between plans and trades, i.e. standardization, not humanoids hanging drywall.
What to watch: whether the eviction-policy literature holds up under the reproducible harness, and how many of this month's KV-cache preprints switch to LRU-leaf as their baseline.
Would you trust a serving-paper benchmark that never showed Belady beating LRU? Tell us in the comments.
Sources: agentic-kv-cache (GitHub) · Hacker News discussion · AgentSysBench (arXiv) · Mooncake (GitHub) · CNBC · NAHB 2026 Housing Outlook
