Jalapeño's first benchmarks put OpenAI on Nvidia's turf

OpenAI's decision to design its own inference chip stopped being a rumor and became a competitive threat today. Fresh off a June announcement, the company published the first real measured results for Jalapeño, the accelerator it co-built with Broadcom — and the numbers, if they hold, are the kind that supply-chain analysts and Nvidia's investor-relations team both lose a little sleep over. The headline "work per watt" gains we flagged in our morning brief are worth sitting with, because the deeper story is not that OpenAI built a fast chip. It's that OpenAI built a chip with its own models, to run its own models — a self-reinforcing loop that no GPU vendor can easily answer.

The numbers, and how to read them

The results, presented at Hot Chips and measured on SemiAnalysis's public InferenceX benchmark, span three very different public models: GPT-OSS 120B, DeepSeek R1 670B, and Moonshot's Kimi K2.5 1T — the largest open model OpenAI tested. Comparing against Nvidia's Blackwell-generation systems, Jalapeño delivered 1.5 to 1.9 times more AI work per watt at peak throughput, and 1.7 to 3.6 times lower end-to-end latency across all three. On interactive, low-latency agent workloads the advantage widened to 2.1 to 4.1 times.

The per-watt framing is deliberate and worth taking seriously. OpenAI stresses that a chip's TDP matters more than a raw per-chip throughput number, because power is what a hyperscaler actually rents. Jalapeño is rated at 700 watts and stayed at or below 550 watts sustained in testing, against 1,200 watts for Nvidia's GB200 and 1,400 watts for the GB300 it was compared with. Roughly half the power, roughly twice the work per watt — at OpenAI's serving scale, that is the difference between paying through the nose for inference and having room to cut prices.

The shape of the gain is architectural, not just a numbers game. Where GPUs generally force a trade-off between prefill (computing a prompt, heavily compute-bound) and decode (generating tokens, bandwidth-bound), Jalapeño keeps a model's working state, including the KV cache, local to the compute that serves it, and lets the network rebalance resources across the rack as a task moves between phases. That is the design that escapes the throughput-versus-latency compromise, and it is precisely the profile agentic workloads punish when it is absent.

The loop that makes this a system, not a chip

The defensible thing for OpenAI here is not the single generation of silicon — it's the pipeline that produced it. Design to tapeout took nine months, which OpenAI calls the fastest ASIC cycle in advanced semiconductors, and that speed is itself an AI product. OpenAI's models helped explore layouts and optimize the arithmetic circuits that make up the compute.

The follow-through is the more consequential claim. OpenAI says its Codex assistant running on its GPT-Astra model brought three open-weight models that were never part of Jalapeño's original production plan to high performance within two months — evidence the software can keep up with new model families rather than bogging down in hand-written kernels. On selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than the human-expert versions they replaced. The vendor caveat applies: those figures cover chosen blocks, not whole models. But the direction is the point. OpenAI increasingly designs the models, the kernels, the serving stack, the chip, the memory, and the networking as one system, then uses its newest models to tighten the loop. Nvidia sells silicon; OpenAI is learning to sell a feedback cycle.

Who wins, who loses, and who's really in the crosshairs

If these results hold up at rack scale, the clearest loser is Nvidia's pricing power — not Nvidia itself, at least not soon. OpenAI says it will keep buying copious Nvidia hardware for both training and inference, and hardware chief Richard Ho has been careful to call Nvidia "a very good partner." But a customer that can serve its own inference volumes on its own silicon, per-watt, is a customer that stops being a price-taker on its largest line item. That is the scenario behind Nvidia's $500 billion data-center pledge with its GPU value guarantee: when your biggest buyers can make their own accelerator, you start guaranteeing their downside on yours.

The structural winners are Broadcom, which did the silicon implementation and networking for a platform it will now license across a multi-generation roadmap, and Celestica, which industrialized the boards and racks. Jalapeño is also broadly a data point for the direction every hyperscaler is already moving — Google's TPUs, Amazon's Trainium, Meta's MTIA, Microsoft's Maia. The GPU "gold rush" gave way to a per-customer custom-ASIC era, and OpenAI is simply the highest-profile believer yet that inference-first silicon can beat the general-purpose alternative on its own economics. That orientation to power is exactly the reason edge and on-device inference is getting serious attention too — see Apple's new Mac Studio and Mac mini being built around local AI — as the industry learns to value work-per-watt over raw speed.

What the skeptics will say

Every asterisk on the release is fair game. These are OpenAI-commissioned numbers on a benchmark its own architecture was clearly tuned toward, tested against Blackwell-generation Nvidia systems while Nvidia's next architecture, the Vera Rubin line Nvidia promotes on a per-watt basis, is already on the way. A purpose-built ASIC beating a general-purpose GPU on power efficiency is, in one sense, the expected result rather than a miracle. The honest question is whether the advantage survives at whole-datacenter scale, with real kernels and real software maturity, once Nvidia ships its own efficiency-leaning generation. And there is the timing: OpenAI says deployment begins in small volumes late this year and ramps through 2027 — a window in which Nvidia gets to respond. Jalapeño also benefits only OpenAI, not the broader industry; rival labs and enterprises still rent from Nvidia either way.

What to watch next

Two things. First, independent reproduction — whether anyone outside OpenAI can get near these per-watt numbers on the same InferenceX runs, and whether they hold once the production fleet is actually serving traffic. Second, the generation race: OpenAI says Gen 2 is already deep in development and Gen 3 is taking shape, while Nvidia's Vera Rubin generation is the first real per-watt counterpunch. If OpenAI can keep the design-to-tapeout cycle short and compound it generation over generation, the field will have permanently relocated the battleground from "who sells the most GPUs" to "who can redesign the accelerator fastest around their own models."

If a first-gen chip that design limits its own silicon is only just beginning to fight for volume, how safe is Nvidia's stranglehold on the datacenter? Tell us in the comments.

Sources: OpenAI — Jalapeño first results · OpenAI — Jalapeño debut with Broadcom · The Verge · TechCrunch · Investing.com