I picked some of the most popular NVFP4 quantizations of Qwen3.8-27B and compared them in quality and speed.

Contestants

These are the models tested in these benchmarks:

Note: I also had RedhatAI's NVFP4 quantization on the list, but then found out that it's actually just a byte-identical copy of the unsloth quantization and therefore dropped it.

Setup

Hardware: single NVIDIA DGX Spark (GB10, 128 GB unified memory)
Engine: vLLM (nightly snapshot)

Every variant was served with an identical config: --max-model-len 262144 (native, no YaRN), --kv-cache-dtype fp8, --gpu-memory-utilization 0.86, MTP speculative decoding {"method":"mtp","num_speculative_tokens":5}, --max-logprobs 256. All generation tiers ran at temperature 0 (deterministic); the agentic tier ran at the deployment regime (temp 1.0, thinking-on).

--gpu-memory-utilization was set to 0.86 for the benchmark (vs. 0.94 in production) to leave headroom for the extra transient memory the logprob-based tiers (PPL/KLD) and the stability probes place on the unified 128 GB.

Benchmarks executed

I benchmarked the five NVFP4 quantizations of Qwen3.8-27B against the official BF16 (reference) and FP8 checkpoints, on a coding/agent-oriented panel (perplexity, distribution fidelity, code generation, math, agentic tool-use, throughput):

Results

Deltas are vs. the BF16 reference (unquantized) model.

Notes:

• ↓ lower is better.
• ↑ higher is better.
• KLD is reported as p50 (median per-position KL), robust to a few high-divergence positions.
• KLD text (49 wikitext prompts) is the apples-to-apples column (clean for every variant).
• KLD code (46 code prompts) is clean for the NVFP4 candidates.
• FP8 code-prompt NaN defect (upstream finding, not a harness issue): the official FP8 checkpoint emits NaN prompt_logprobs on code-like text (14 NaN + 7 extreme + 18 elevated, all code; text clean). Generation is unaffected. Consequently FP8 KLD code is N/A; its KLD text (0.0117) confirms near-lossless.
• MTP acceptance is a short-sample health-gate estimate; the operator's real-time dashboard is authoritative. All variants show healthy acceptance (0.88–0.96).
• Codegen scoring was validated against ground truth (correct solutions pass, broken ones fail). MBPP prompts are given the function signature (the dataset's natural-language prompts omit it).
• PPL is measured per-variant with each variant's own tokenizer (standard); the wikitext-2 test split is the corpus.
• unsloth's KLD is valid despite a different tokenizer file. Its tokenizer's core vocabulary is byte-identical to the reference (verified 7121/7121 positions align), so its KLD is comparable.

Fidelity Rankings

Closest to the BF16 reference first (ranked by KLD text, the most sensitive metric; PPL shown for reference):

Speed Rankings

Fastest first (single-stream, small-context decode; min / mean / peak):

The ordering is stable across min / mean / peak — NVIDIA leads on all three.

Bottom line

• All five NVFP4 quants are near-lossless on task accuracy — within ~1–2 points of BF16 on MBPP, HumanEval, GSM8K, and agentic tool-use.
• unsloth is closest in quality to reference BF16 (best PPL and KLD) and 3rd fastest.
• NVIDIA is now fastest (38.6 tok/s).
• RadixArk (NVFP4 LMHead) is 2nd fastest but farthest in quality from BF16.
• The two RadixArk models (LM-head ablation): Quantizing the LM head to NVFP4 (vs keeping it BF16) costs ~0.1 PPL and ~0.04 KLD-text but buys ~12 tok/s (34.6 vs 22.7) and ~1.8 GB. A clear accuracy↔speed tradeoff localized to the output head.
• Speed is not purely size-driven. The NVFP4 quants (19–22 GB) run 1.7–2.6× faster than BF16 (51.77 GB). Within the NVFP4 set, NVIDIA (21.92 GB) is fastest (38.6 tok/s) and QUASAR (19.17 GB, the smallest) is just 4th (25.4 tok/s) - so speed scales with checkpoint size and kernel path (i.e. the kernel path and mixed-precision recipe also matter).

Recommendation

The numbers narrow it to two clear leaders; the other NVFP4 builds are mid-pack and are not the best on any axis.

• unsloth NVFP4 — the overall leader. Best fidelity (closest to BF16 on PPL +0.138 and KLD 0.0475 text / 0.0197 code), competitive on all task metrics, and 3rd-fastest (31.2 tok/s). The default pick when fidelity matters most.
• NVIDIA — the speed leader. Fastest (38.6 tok/s), while yet retaining excellent fidelity and decent overall quality results.

So it is that simple, right? Not quite. Because during my tests I encountered some stability issues. The unsloth model appears to suffer from numerical instability, causing issues like garbled / gibberish output under rare situations. Furthermore all NVFP4 quantizations appear to be prone to reasoning loops, some maybe more often than others, but I could not clearly tell which one is better or worse on this regard yet. It would make sense to run a separate benchmark that just covers these stability issues. In general I have the impression though, that this Qwen3.8-27 base model is much more sensitive to NVFP4 quantizations than the previous generations.

Note: these were taken under the assumption of using MTP (speculative decoding). If you don't want to or cannot use MTP, then the picture might be different. I have not tested without MTP, because in production MTP is typically enabled and is a huge performance lever, without sacrificing quality (equal sample distribution).

History:

• 2026-09-09: Initial test results (four NVFP4 quants: unsloth, RadixArk (BF16 LMHead), RadixArk (NVFP4 LMHead), QUASAR-QAT).
• 2026-09-12: Added results of Nvidia NVFP4 quantization, and fidelity/speed ranking tables.