leoncca/Qwen3.8-Flash-Next-AWQ-g32

Qwen3.8 Flash Next AWQ g32 + QSA FP8 E4M3 KV

AWQ derivative of Qwen/Qwen3.8-Flash-Next. Only routed-expert projections are quantized. PLE is reused from the official FP8 checkpoint, and calibrated QSA FP8 E4M3 K/V scales are included.

138.13 GB across 44 model shards: 10 AWQ, 33 PLE, and one K/V-scale shard.

Checkpoint

• Base revision: f5d08274bafd880402bd16f5e3e6c514136ec06c
• PLE revision: bcd9f01ddc9cff2316eb84281bebcd5b058bddce
• Routed experts: asymmetric AWQ W4A16, g32, zero point, GEMM layout
• Quantized: per-expert gate_proj, up_proj, and down_proj
• Kept at source precision: vision, PLE, attention, routers, shared experts, embeddings, LM head, hyper-connections, and MTP
• Indexed tensors: 222,771 with 137,042,968,666 bytes of tensor payload
• Merged index SHA-256: f066a0154a9101b359c3a4d4fa6a83fb12b7126a9cf0f19cb772611d08cc07ee

Machine-readable details are in EXPERT_COVERAGE.json, MODEL_PROVENANCE.json, VALIDATION.json, and SHA256SUMS.

Why AWQ g32

Two properties favor asymmetric AWQ here:

1. On hardware without native FP4 execution, AWQ W4A16 provides a practical weight-only path while keeping activations in FP16.
2. The routed-expert weights frequently have off-center local ranges, which an asymmetric affine codebook can represent with a movable zero point.

A route-frequency-stratified CPU audit covered all 93 layer/expert pairs with fewer than 128 natural tokens, including all eight zero-hit pairs, plus controls from every layer. In total it compared 707 layer/expert pairs, all three routed projections, 2,121 matrices, and 69,500,928 BF16 weight values. It dequantized the actual NVFP4 checkpoint and compared both formats against the same BF16 mother weights; all 48 unquantized router tensors matched bitwise.

Route stratum	Pairs	Matrices	Actual NVFP4 relative RMSE	Affine g32 relative RMSE	Reduction	g32 wins
Zero natural tokens	8	24	0.094704	0.082320	13.08%	24/24
1–31 natural tokens	14	42	0.094387	0.083253	11.80%	42/42
32–63 natural tokens	17	51	0.094737	0.082029	13.41%	51/51
64–127 natural tokens	54	162	0.094747	0.082104	13.34%	162/162
Complete <128 census	93	279	0.094685	0.082289	13.09%	279/279
Non-low-frequency controls	614	1,842	0.094887	0.081588	14.02%	1,842/1,842
All deduplicated selections	707	2,121	0.094859	0.081686	13.89%	2,121/2,121

Affine g32 had lower error in every sampled matrix, including all 279 low-frequency gate/up/down matrices. The complete low-frequency census also reduced NMSE by 24.47%.

The result was not only a group-size effect: asymmetric affine g16 reached 0.069948 relative RMSE, compared with 0.094859 for actual NVFP4. Of the sampled BF16 groups, 56.78% had a range-center shift above 0.1 and 25.00% above 0.2. Only 55.40% of raw affine g32 zero-points and 58.73% of released AWQ zero-points landed at the central 7/8 positions. These weights benefit from a movable asymmetric zero point rather than a fixed zero-centered codebook.

The g32 layout also matches the model's width-640 experts under TP4: 640 / 4 = 160, and 160 is divisible by 32 but not 128. This avoids storing g128 metadata that must later be expanded to effective g32. The cost is about 6.59 GiB more scale and zero-point metadata than g128.

Calibration coverage

AWQ weight calibration used 684 records and 202,750 active tokens, producing 97,320,000 native top-10 token-to-expert assignments across 48 layers. Natural routing covered 24,568 of 24,576 layer/expert pairs (99.9674%), with 45/48 layers at 512/512. The remaining eight zero-hit pairs and 85 low-coverage pairs were explicitly augmented for gate/up/down projection calibration. The released checkpoint requires zero runtime fallback pairs. This corpus is separate from the 4,130,597-token E4M3 K/V-scale calibration described below.

Loading requirements

The loader must support:

• Qwen4Exp per-expert asymmetric AWQ W4A16 g32 in GEMM layout;
• indexed reuse of complete PLE shard files while ignoring unindexed tensors;
• all 24 bundled, finite, positive QSA K/V scale entries;
• complete Qwen4Exp multimodal support, including MRoPE and video-token-pruning initialization, for image and video inference.

The image-text-to-text tag requires the complete multimodal path; text-only loading does not establish image or video support.

Validation

Validation results — updated 2026-09-09

Each KV format was evaluated on 47 text prompts and 50 supplementary checks, with thinking enabled throughout. These are small-sample regression checks, not full benchmark scores. Repeated requests are not additional unique tasks.

Quality check	FP16 KV	Calibrated E4M3 KV
GSM8K five-shot subset	32/32	32/32
HumanEval functional subset	5/5	5/5
MBPP sanitized functional subset	5/5	5/5
IFEval strict, prompts	5/5	5/5
IFEval loose, prompts	5/5	5/5
IFEval strict / loose, instructions	Both 12/12	Both 12/12
Basic generation (two four-request sequences)	8/8	8/8
Needle retrieval, six lengths from 1K to 128K	6/6	6/6
128K needle retrieval, three positions × two repeats	6/6	6/6
Image requests	10/10	10/10
Video requests, including own-answer follow-up	8/8	8/8
Image/video total	18/18	18/18
Tool workflow success	Not established; 12 requests captured	Not established; 12 requests captured
128K retrieval exact-repeat stability	3/3	3/3
Image/video exact-repeat stability	8/9	8/9

GSM8K note (case gsm8k-0012, Carlos's lemon tree): $90 / ($10.50 − $3) = 12 years to recover the tree's cost, with profit thereafter. Thus 12 is a valid elapsed-wait answer (32/32); 13 names the first profitable year, not the number of completed years to break even.

Settings: native 262,144-token shared input + reasoning + final context, no YaRN, xhigh thinking, and the full remaining context minus 16 safety tokens for generation; no separate reasoning cap. The 47 text prompts used temperature 1, top-p 0.95 and top-k 20; supplements used greedy decoding (temperature 0, top-p 1, top-k −1). Both used seed 0, min-p 0, presence/frequency penalties 0 and repetition penalty 1. Only final responses or emitted tool calls are scored. Tool proposals were captured but not executed. These redacted traces have abbreviated tool schemas and placeholder attachment paths, and the recorded next action is not necessarily unique. Name matching cannot establish whether the task was solved, so no tool-task success score is claimed. Video follow-ups use that run's own preceding answer; an unavailable valid parent is recorded as a failed, unattempted follow-up. Multimodal answers, including raw passes, were reviewed against the media and question wording. Unique-color spatial references and equivalent direction phrases are accepted; requested answer order and explicit formatting still apply. Exact-repeat stability describes identical outputs, not correctness: different wording can express the same correct answer.

Runtime: four 32 GiB V100 GPUs, TP4, FP16 compute, MTP off, compilation mode 3, FULL_AND_PIECEWISE CUDA graphs, prefix caching and chunked prefill, maximum two sequences and 8,192 batched tokens; regression requests issued sequentially. Source revision: 752f86495f9560280896d1fccecc60c8d49bc7b4.

Supplement completion status (FP16): 0 output-cap hits, 0 transport-incomplete or unattempted cases, including 0 unavailable follow-ups; 0 tool requests with neither a final answer nor an emitted call. Supplement completion status (E4M3): 0 output-cap hits, 0 transport-incomplete or unattempted cases, including 0 unavailable follow-ups; 1 tool requests with neither a final answer nor an emitted call.

The retests support enabling thinking and leaving ample generation space to better realize this model's capabilities; historical settings also differed, so improvements cannot be attributed to thinking alone. No same-mother BF16 reference was run. MBPP task 617 retains its fractional-jump test inconsistency; passing it establishes test compatibility, not a general jump-count solution.

The E4M3 scales were calibrated from 4,130,597 tokens. All 24 values are finite and range from 0.0171072837 to 0.0806361660, with zero observed calibration saturation. First tokens matched on 18/18 checks. Selected-block recall averaged 0.995906 with a minimum of 0.991822; QSA outputs had minimum cosine 0.998597 and maximum relative L2 0.052962.

The multimodal set contained five image and four video cases, each repeated twice. All 333 model.visual.* tensors are bitwise identical to the reference checkpoint, so AWQ did not alter the vision tower payload.

Notes

• The bundled K/V scales belong to this exact merged checkpoint and should not be replaced with scales from another weight artifact.
• The reported results are regression checks rather than a full capability benchmark.
• This derivative uses the included Qwen Community License 1.0.

Integrity

Verify the repository with SHA256SUMS. The authoritative artifact identity is the merged index SHA-256 shown above.

Optional FP8 MTP

An independently converted, matching draft is available in mtp-fp8/. It stores only MTP routed experts in FP8 E4M3 with explicit block scales; other draft parameters retain source precision. Select that directory explicitly as the speculative draft with modelopt_mixed in a compatible Qwen4Exp MTP runtime. The main AWQ weights, indexed default MTP, and bundled KV scales are unchanged. A full unfiltered repository download also includes this optional 4.88 GiB tensor payload.

The quality results above describe the original configuration; they do not evaluate this optional FP8 draft. Its bounded loading/generation checks and loading requirements are documented separately in the option directory.