Inspur says one box now runs 2.8T Kimi K3 on 128 chips

The largest open-weight model in existence has been given a machine that fits in one rack — and there is no Nvidia silicon inside it.

What Inspur actually put on stage

At the AI Computing Conference in Beijing on September 21, Inspur Information launched the Yuanbrain SD200 Ultra supernode alongside a second product, the Yuanbrain HC2000 multi-compute rack. The claim for the SD200 Ultra is compact: it runs Moonshot's Kimi K3 — 2.8 trillion total parameters, 104 billion active per token — on a single machine assembled from 128 domestic AI chips, at under 5.85 milliseconds per generated token. That works out to roughly 170 tokens a second for a single user, which Inspur describes as five times the industry average. It also says the same box will hold frontier models up to 10 trillion parameters.

The architecture underneath is where the substance sits. Inspur calls it 3D Hyper Mesh, and the numbers are a capacity plan rather than a compute pitch: 8 TB of unified addressable accelerator memory, 64 TB of system memory, and a fabric Inspur measures at 0.69 microseconds of communication latency. Symmetric memory lets one accelerator read another's memory directly instead of copying through a host, which Inspur says cuts AllReduce time by a factor of 3.5. The chips are not named beyond "domestic," which is the first thing a sceptic should write down.

The software claim is the more interesting one, because Inspur says it built it with the model it is serving. For K3 inference the company used Kimi K3 itself as a "super-operator agent" to fuse the base operators inside KDA, gated MLA and MoE into larger units with tile-level pipelining, reporting a tenfold drop in operator count and more than three times the inference throughput. A year ago the comparable Inspur machine was the plain SD200: 64 domestic chips, 4 TB of memory, running Kimi K2 at 1.2 trillion parameters.

Why the box, not the chip, is the product

Kimi K3's own deployment guidance is the context that makes this launch legible. The weights are roughly 1.56 TB of MXFP4 across 96 shards, and Moonshot recommends deploying on supernode configurations with 64 or more accelerators. Mixture-of-experts sparsity buys you compute per token — only 16 of 896 experts activate — but it does not buy you memory. Every expert has to be resident and reachable, because the router can call any of them on the next token. For a 104-billion-active-parameter model, the binding constraints are how much fast memory one coherent domain can hold and how quickly tokens can cross between chips. Peak floating-point throughput is close to irrelevant.

That is the argument for scale-up over scale-out, and it is a strategic argument as much as a technical one. A country whose access to leading-edge process nodes and high-bandwidth memory is restricted cannot win by making a better single chip. It can plausibly win by making 128 ordinary chips behave as one machine with sub-microsecond links, which is exactly the thing the SD200 Ultra's fabric is priced to deliver. Inspur's own framing splits the market into Capability AI compute — what the strongest single system can do — and Capacity AI compute, meaning how much intelligence you can produce per unit of capital. Its bet is that the second category is where the volume is, now that agent workloads run for hours and hold long key-value caches.

We have tracked both halves of this race. The rented-Nvidia path showed up in August when a developer served the same model on eight B300s — Kimi K3 fits on 8 B300s — 92 tokens/s at $190 a million — and the Chinese cloud-supernode path arrived the same month with Alibaba Cloud's Zhenwu M890 supernode runs 2T-parameter models. Inspur's machine is aimed at the gap between them.

Where the numbers are soft

Three things to hold apart from the announcement. The first is that every performance figure here is a vendor number delivered at a vendor's own conference, with no third-party benchmark and no named baseline. "Five times the industry average" and HC2000's "ten times the token capacity for the same investment" are both ratios to something Inspur has not identified; a ratio without a denominator is marketing. The comparison to that eight-B300 deployment has to be made carefully too: the B300 run was a community configuration under vLLM on rented hardware, Inspur's figure comes from a tuned stack with fused operators, and both are single-stream numbers that say little about cost per token under load.

The second is arithmetic worth doing on Inspur's own disclosure. Eight terabytes spread across 128 accelerators is roughly 64 GB of memory per chip. Nvidia's Blackwell Ultra carries 288 GB per GPU. So the SD200 Ultra wins on the capacity that matters for a 1.56 TB model with a long context — and loses by a wide margin on memory density per chip. Aggregate eight B300s and you have about 2.3 TB in one node; Inspur is offering 8 TB in one domain, but it takes 16 times the silicon to do it. Whether that trade is good depends entirely on the price of a domestic accelerator, and Inspur did not give one.

The third is that this is a listed company in the middle of a capital raise aimed at AI infrastructure and liquid cooling, announcing at an event with thousands of industry attendees. That does not make the engineering wrong. It does mean the numbers arrive with an audience and a share price attached, and the honest reading of a launch like this is that the architecture is the product and the latency is the promise. One more gap: Inspur's 10-trillion-parameter claim has nothing to run on it yet — the largest open weights shipping today are less than a third of that.

What to watch

Name the chips. A 128-way coherent domain at 0.69 microseconds is an impressive systems result only if the parts are what we think they are, and Inspur has left that blank. Then watch for reproduction: someone outside the company serving Kimi K3 at full million-token context and reporting the latency without a vendor's tuning harness. And watch the cost line, because it is the one that decides the story — if the domestic box cannot beat roughly $190 per million output tokens against rented Blackwell, then a supernode that runs open weights without an export licence is a sovereignty purchase rather than an economic one, and the capacity market Inspur is describing stays where it is.

If domestic silicon can run the largest open model in one box but costs more per token than rented Nvidia, which number wins? Tell us in the comments.

Sources: ITHome (via iFeng) · Jiemian News · Zhidx · MIT Technology Review China · Kimi K3 technical report (arXiv)