---
license: mit
base_model: zai-org/GLM-5.3-BF16
base_model_relation: quantized
tags:
- glm
- w4afp8
- quantized
- moe
pipeline_tag: text-generation
---

# GLM-5.3-W4AFP8

W4AFP8 quantization of [zai-org/GLM-5.3](https://huggingface.co/zai-org/GLM-5.3), quantized
from the [BF16 master](https://huggingface.co/zai-org/GLM-5.3-BF16): MoE expert weights in
INT4 (group-128, AWQ-calibrated), activations and non-expert layers in FP8, served by
SGLang's native CUTLASS W4A8 grouped GEMM on Hopper (SM90). Roughly 2x the KV-cache
capacity of the FP8 checkpoint at identical serving throughput, with EAGLE/MTP speculative
decoding intact.

Also mirrored on [ModelScope](https://modelscope.cn/models/PhalaCloud/GLM-5.3-W4AFP8) (魔搭)
for faster downloads in China.

Successor to [PhalaCloud/GLM-5.2-W4AFP8](https://huggingface.co/PhalaCloud/GLM-5.2-W4AFP8);
GLM-5.3 is architecturally identical to GLM-5.2 (verified tensor-by-tensor), so the same
serving configuration applies.

## What is different from the 5.2 checkpoint

- Quantized from the BF16 master (`zai-org/GLM-5.3-BF16`), not the FP8 release, avoiding
  double quantization of expert weights.
- AWQ calibration on coding-agent traces (SALT-NLP/SWE-chat, 128 windows x 2048 tokens,
  session-disjoint split), matching the production agentic workload, instead of generic web
  text.
- The MTP/nextn draft layer (layer 78) ships quantized; EAGLE speculative decoding works
  out of the box.

## Evaluation

Measured on this checkpoint served with the SGLang configuration below
(official request parameters, reasoning at the chat-template default).

| Benchmark | GLM-5.3-W4AFP8 | GLM-5.3 reference |
|---|---|---|
| GPQA-Diamond | **91.9** | 91.7 |
| AA-LCR | 73.0 | 76.3 |
| BFCL (45-item live subset) | 82.2 | — |
| NIAH retrieval @ ~930k-token prompts | **3/3** needles (depths 0.1/0.5/0.9) | — |
| Teacher-forced dNLL vs BF16 (held-out coding traces) | +0.28 nats | — |
| Speculative accept length (EAGLE steps=3, topk=1, draft=4) | ~2.9 | — |

References: [Artificial Analysis](https://artificialanalysis.ai/models/glm-5-3) GLM-5.3.
Per-item artifacts are preserved in the [repository history](https://huggingface.co/PhalaCloud/GLM-5.3-W4AFP8/tree/d5f180e842f22b142255bd8d5760f4145fe0036c/aux/results).

## Serving

```bash
python -m sglang.launch_server \
  --model-path PhalaCloud/GLM-5.3-W4AFP8 \
  --tp 8 \
  --quantization w4afp8 \
  --disable-shared-experts-fusion \
  --speculative-algorithm EAGLE \
  --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 \
  --reasoning-parser glm45 --tool-call-parser glm47
```

H200-class GPUs serve the full 1M context; on H100 80GB use `--mem-fraction-static 0.78`
and a reduced context length. vLLM support for this exact checkpoint format is pending
upstream in vllm PR #53848 (`--quantization w4afp8` via the humming backend).

## Verification

- Structural audit: tensor inventory (176,997 tensors) matches the format-proven
  GLM-5.2-W4AFP8 checkpoint exactly (names, dtypes, shapes).
- Dequantization audit: int4 expert weights reconstruct within the expected group-128
  envelope (10.8-13% relative RMS residual after AWQ channel-scale fitting; no outliers).
- Held-out teacher-forced NLL and task benchmarks: see Evaluation above.

The per-layer AWQ search checkpoints (scales and clip thresholds for all 78 layers) are
preserved in the [repository history](https://huggingface.co/PhalaCloud/GLM-5.3-W4AFP8/tree/d5f180e842f22b142255bd8d5760f4145fe0036c/aux) for reproducibility and further research;
they are kept out of the current revision so `snapshot_download` fetches only the
serving weights.