Commerce ordered Kalshi to pull its AI compute price tracker
Two stories today about where AI's bottlenecks actually are: who is allowed to put a price on compute, and whether an agent that works today will work tomorrow.
The US Commerce Department ordered Kalshi to take down its AI-compute forward curve last month, citing national security concerns, and Kalshi quietly complied. The product was not a futures contract in the ordinary sense — it pulled together data from several markets where users bet on the cost of renting Nvidia chips and published an overall curve of where compute prices were heading. Semafor's Reed Albergotti and Liz Hoffman reported the order, and noted that many of the underlying markets Kalshi runs remain open for trading. Representatives for Commerce and Kalshi both declined to comment; the CFTC did not respond to a request for comment.
Why an administration this enthusiastic about AI would shut down a price index is the part worth sitting with. Compute cost has quietly become one of the most load-bearing numbers in the US economy: older chips serve as collateral for billions of dollars of borrowing by neoclouds like CoreWeave and underpin data-center deals, while the companies buying AI worry that power and infrastructure shortages will send token prices soaring. Market participants gave Semafor one plausible motive — a compute curve can be pushed to show a sharp drop in the value of older chips, and that number would land directly on AI stocks and debt. Some of these markets are thinly traded, so a bad actor isn't strictly required for volatility. Either way, the department treated the market's output as a national-security object rather than a neutral reading.
Separately, Commerce pushed the CFTC to effectively freeze approvals of new compute contracts for 60 days, according to the same reporting, a pause that would reach well beyond Kalshi. The CFTC's own August request for comment on listing compute derivatives is the public paper trail, with comments open for 60 days after Federal Register publication. Its chairman, Michael Selig, framed the exercise as a foundation rather than a brake, saying the agency would do for compute "what American markets did for the commodities that powered the industrial economy." The timing is what stings for the industry: CME and Silicon Data had set an October 5 launch for H100 and B200 rental-index futures, ICE has been exploring its own products, and Architect Financial Technologies was moving on the same idea. We covered the CME plan when it landed — CME to launch GPU rental futures as AI compute goes tradable — and the data layer underneath it when the pricing startup raised — Silicon Data raises $30.5M to price the GPU futures market. A commodity market that only works when its prices are flattering to the incumbent trade is not a commodity market; it is a mood ring with a compliance department.
IBM Research has a fix for the agent that impresses in the demo and then takes a different path in production — and the diagnosis is more useful than the fix. Most agent benchmarks report the average: on AppWorld's normal test split, a ReAct agent running GPT-4.1 succeeded on 77.4% of runs across five repetitions. But it succeeded on all five runs for only 53.0% of tasks — a 24.4-point gap between "how good is it on average" and "will it still work if I ask again," which widens to roughly 30 points on the hard tier. The authors call that the consistency gap, and their Consistency Analyzer finds it by resampling each decision step inside a single recorded trajectory, with no ground truth and no second end-to-end rollout required. It is one extra model call per decision point, asking for five completions instead of one, and it flags where the answer was nearly a coin flip.
The mechanism matters because the usual tools don't touch it. The runs happened at temperature zero, so this isn't ordinary sampling noise; decisions come out of probability distributions, and flat distributions can be reordered by tiny platform-level nudges like request batching. Greedy decoding and fixed seeds govern how a distribution becomes a token, not the shape of the distribution itself — a capricious agent is a separate axis from a weak one, and a bigger model won't automatically fix it. Feeding the flagged steps back as guidelines cut the gap from 24.4 points to 12.0, lifted all-five-runs success from 53.0% to 69.0% and raised average accuracy from 77.4% to 81.0% rather than trading it away. The guidelines also transferred to a different task in the same scenario at +13.0 points, and on the weaker gpt-oss-120b the similar-task gain beat the same-task gain, which is the signature of reusable failure patterns rather than memorized trajectories. The analyzer and guideline generation now ship in the open-source ALTK-Evolve toolkit, which is the fastest way to test whether your own agent's failures are load-bearing or luck. We saw the same axis from the other end earlier this month, when thousands of runs showed coding agents disagreeing on tool choices — 17,000 agent runs: coding agents disagree on tools 58% of the time.
What to watch: whether CME's October 5 compute futures launch survives the 60-day comment window, and whether any agent vendor starts publishing a consistency number next to its accuracy claim.
Would you trust an agent that gets your task right three times out of five? Tell us in the comments.
Sources: Semafor · CFTC Release 9286-26 · Kalshi compute forward curves · Techmeme · IBM Research · arXiv technical report · ALTK-Evolve (GitHub)
