DFlash 2 brings 20% faster parallel drafting to the inference stack
Inference is eating AI's bill, and a new open-source release just made it cheaper to run.
DFlash 2, a block-diffusion draft model from Inco AI, ships checkpoints for Qwen3.8-27B and Muse-Glimmer-30B — and claims over 20% more output per verification pass at roughly 1% added latency. The core idea is speculative decoding: a small draft model guesses a block of tokens, the large model verifies the whole block in one forward pass. DFlash made that guess itself parallel (one pass, not one token at a time); DFlash 2 pushes it further by keeping each position's top candidates and tracing one coherent path through them. Across benchmarks the acceptance-length gain runs 16–25%, and on Qwen3.8-27B it hits the throughput of autoregressive decoding at batch size one in SGLang — meaning you get more tokens per second without burning extra GPU cycles. The release lands at a moment when agents are the dominant consumer of inference tokens: they read, plan, and call tools for hours, and every token costs a full forward pass. NVIDIA measured DFlash 2 on Blackwell GPUs; Google reported three times more tokens per second on TPUs with DFlash; CoreWeave's production Kimi K2.7 Code endpoint runs DFlash by default. On Hugging Face, DFlash models have passed 3.5 million total downloads. The checkpoints are live now — DFlash 2 drafts for Qwen3.8-27B and Muse-Glimmer-30B on Hugging Face, with integration in SGLang, vLLM, llama.cpp, and oMLX. For teams running local or self-hosted models, the math is simple: more tokens per verification pass means lower cost per agent-hour, which is the number that matters as workloads shift from chat to long-running tasks.
What to watch: whether DFlash 2's gains hold up as models scale past 30B parameters, and how quickly inference engines bake it into their default paths.
Does speculative decoding change how you think about running agents locally? Tell us in the comments.
Sources: DFlash 2 GitHub (z-lab)
