Qwen3.8-27B ships a fully NVFP4 build that keeps its benchmark scores at a third of the size
A quantization team called QUASAR-QAT has released a fully 4-bit build of Qwen3.8-27B that keeps nearly all of the original model's accuracy — and it did it by retraining the quantization in, not by rounding weights after the fact.
A fully NVFP4-quantized checkpoint of Qwen3.8-27B is out, and it posts scores within a rounding error of the unquantized original while cutting the download from roughly 56 GB to under 20 GB. The release comes from a group publishing under the name QUASAR-QAT, and the interesting part is the method: instead of taking the finished BF16 model and squeezing it after the fact — the usual post-training route, which is why aggressive 4-bit builds usually lose a few points on hard benchmarks — they trained the quantization itself. Using what they call quantization-aware distillation with their QUASAR algorithm, the compact model was distilled from the original full-precision model as its teacher over 2,446 steps until it learned to live inside the cramped 4-bit format.
The numbers explain why the r/LocalLLaMA thread around the release took off. On GPQA-Diamond, a graduate-level science reasoning benchmark, the fully quantized model scored 0.9091 against the original's 0.9141; on AIME 2026 math problems it matched the original's perfect 1.0000 across repeated runs. Every linear layer in the network is quantized to the 4-bit format — normally the point where aggressive builds fall apart, because teams keep attention layers at higher precision precisely to avoid the damage. The comparison table also puts existing post-training 4-bit builds to shame: community checkpoints from Unsloth and Inferact score lower (roughly 0.894 and 0.876 respectively on GPQA-Diamond) at larger file sizes.
Why it matters: this is the compounding story of small open models becoming genuinely deployable, told one technique at a time. Qwen3.8-27B is barely two weeks old and already the most-hacked-on local model in the community — we covered the debut in Qwen open-sources Qwen3.8-27B that runs on consumer GPUs, the million-download demand curve in Qwen3.8-27B hits 1M downloads as small open models close the frontier gap, and the speculative-decoding speedups in DFlash 2 doubles Qwen3.8-27B speed in llama.cpp — on one GPU. A third of the memory for the same answers changes the calculus again: the model now fits comfortably on a single 24 GB card with long context intact, and the checkpoint serves directly on vLLM with Blackwell-generation Nvidia GPUs. Our take: distillation-recovered quantization keeps showing up as the difference between a demo and something you ship, and this is one of the cleanest demonstrations yet at the 27B scale — the technique traces back to Nvidia's published work on quantization-aware distillation for the same 4-bit format, so expect other open models to get the same treatment soon.
One honest caveat before you clear disk space: the gains are measured on the benchmarks the authors chose, the comparisons come from their own evaluation runs, and independent replications are only starting to surface. Early community reactions are enthusiastic but thin — questions about whether the format can be converted for GPUs without native support are still open in the thread.
What to watch: whether independent replications confirm the near-BF16 scores, whether a llama.cpp path emerges for cards without native 4-bit support, and which open model gets the QUASAR treatment next.
If a 4-bit rebuild really costs you nothing in accuracy, what excuse does your next project have for calling a hosted API? Tell us in the comments.
Sources: QUASAR-QAT release announcement (Reddit r/LocalLLaMA) · QAD: Quantization-Aware Distillation for NVFP4 (arXiv) · Qwen3.8 27B QUASAR NVFP4 (LLM Explorer) · NVIDIA developer forums: Qwen3.8-27B-NVFP4 measurements
