Kimi K3 fits on 8 B300s — 92 tokens/s at $190 a million
The 2.8-trillion-parameter open model that most GPU fleets can't hold turned out to be hostable on a single rack — and the community is now arguing over what that really costs.
A developer posted a full end-to-end deployment of Kimi K3 on eight NVIDIA B300 accelerators rented through Modal, running vLLM with tensor-parallel 8 and native MXFP4 quantization. The rig serves the massive open model at a steady 92 tokens per second with about a one-second time to first token, and the headline economics came out to roughly $190 per million output tokens on a single-stream basis. The model's 1.561 TB of MXFP4 weights across 96 shards take a 27-minute cold boot to load, and leaving the box warm runs about $1,363 a day.
The same write-up tried the cheaper path too: Unsloth's 1-bit dynamic quant, which packs Kimi K3 down to a 594 GB footprint that fits on far more modest hardware — eight A100-80GBs rented for $19.99 an hour. That route gets about 9 tokens per second and lands around $620 per million output tokens, roughly 3.3x pricier per token. The author notes 1-bit quality stayed serviceable — correct arithmetic and coherent prose — but the tradeoff in speed and cost is stark.
The compelling part is the debate in the replies, because the "$190 a million" figure is really a single-stream number. Several commenters pointed out that real inference providers profit by serving many users in parallel across those same eight GPUs, which changes the per-token economics dramatically — the money only makes sense at high throughput, not for one lonely chat session. There's also healthy skepticism that the cost math hand-waves the amortized hardware and the reality that a 2.8T model needs that much silicon regardless of utilization.
The takeaway is a genuinely useful signal for anyone weighing how close we are to frontier-class models in private deployments: a 2.8-trillion-parameter open model is no longer theoretical hardware trivia — it now lives in a documented, reproducible config with real numbers. The catch is that at one user at a time, frontier inference is still roughly an order of magnitude out of the hobbyist budget, and the "cheap" quant route trades speed for a higher per-token bill. As better quantization methods and cheaper accelerators keep improving, that ceiling is moving fast.
Could a 2.8T-parameter model ever become a plausible private deployment for a small team, or will frontier inference stay a shared-cloud game? Tell us in the comments.
Sources: LocalLLaMA — Kimi K3 hosted on 8 B300s · Vizuara — How to Host Kimi K3
