Qwen3.8-27B-DSpine-b16 for MLX

Tencent Hunyuan's DSpine drafter for Qwen3.8-27B, ready to run on Apple Silicon with dspine-hybrid-mlx.

DSpine is a parallel drafter. A small 5-layer model (2.0B parameters) proposes a block of tokens, and Qwen3.8-27B checks the whole block in one pass. You get the 27B model's own output, faster.

Qwen3.8-27B is a hybrid model, with 48 GatedDeltaNet (linear-attention) layers and 16 full-attention layers. Support for it, with exact rollback of the linear-attention state and the drafter's 8192-token sliding window, is now merged into Tencent's own dspine_mlx (Tencent/DSpine#3). You can run this drafter with either package and any MLX build of Qwen3.8-27B.

The weights are Tencent's bf16 checkpoint, unchanged (revision ea909c0), in the safetensors layout MLX loads directly.

Quick start

pip install "git+https://github.com/internet-projects/dspine-hybrid-mlx.git"
dspine-hybrid-generate \
  --target mlx-community/Qwen3.8-27B-4bit \
  --draft netprojects/Qwen3.8-27B-DSpine-b16-MLX \
  --prompt "Write a Python function that checks whether a number is prime." --baseline

--baseline also runs plain decoding and prints the speedup. Add --adaptive for the fastest greedy decoding with thinking off.

To use Tencent's package instead, clone Tencent/DSpine, run pip install -e dspine_mlx and pass this repo as the draft. dspine-hybrid-mlx remains the one-line install and adds --adaptive.

Results (MacBook Pro M5 Max, 128 GB, v0.2)

Decode speed with and without DSpine

Greedy, compared with plain mlx-lm on the same target. Thinking on, 512 new tokens, 10 prompts (2 each from GSM8K, MATH-500, HumanEval, MBPP and MT-Bench), default settings (4-bit drafter):

Target build Plain mlx-lm With DSpine Speedup Verify block
4-bit (mlx-community/Qwen3.8-27B-4bit) 31.9 tok/s 64.7 tok/s ร—2.03 5
8-bit (mlx-community/Qwen3.8-27B-8bit)ยน 17.9 tok/s 52.1 tok/s ร—2.91 5
bf16 9.1 tok/s 39.1 tok/s ร—4.31 16

ยน Measured on a local conversion with the same settings as mlx-community/Qwen3.8-27B-8bit (8-bit, group 64, affine), made with mlx_lm.convert --hf-path Qwen/Qwen3.8-27B -q --q-bits 8 --q-group-size 64.

Decode speed with thinking off

Thinking off: 4-bit 32.0 โ†’ 71.0 tok/s (ร—2.22), 8-bit 18.0 โ†’ 54.0 tok/s (ร—3.00), with --adaptive.

Long prompts with and without DSpine

Long prompts (thinking off, 256 new tokens): with a 33k-token prompt, 4-bit 28.3 โ†’ 57.3 tok/s (ร—2.02) and 8-bit 16.7 โ†’ 47.7 tok/s (ร—2.85). Peak memory 29 GB and 42 GB.

Why block 5 and a 4-bit drafter. On an Apple GPU a quantized 27B forward pass costs about the same for 1 to 5 tokens and then steps up (4-bit: 34 ms for 1 token, 42 ms for 5, 89 ms for 16). Checking 5 drafted tokens per round accepts fewer of them than checking 16, but each round is much cheaper. The drafter runs once per round, and a 4-bit drafter proposes almost the same tokens as the bf16 one (3.40 vs 3.35 accepted per round). Use --block-size and --draft-bits (8, or 16 for bf16) to change them.

Acceptance and quality (thinking off, bf16 drafter, full 16-token block). These are not the conditions of the speed tables: short math answers reach about 7.2 tokens per round. Greedy, first 50 GSM8K test questions:

Target build Tokens per round GSM8K accuracy
4-bit 7.25 90%
8-bit 7.32 84%
bf16 7.33 86%
Tencent's model card (GSM8K, T=0) 7.20

The output is the target's own greedy output, up to bf16 rounding at exact near-ties. See the GitHub README for the checks.

Files

File Notes
model.safetensors DSpine drafter, bf16, 4.05 GB (Tencent revision ea909c0, unchanged)
config.json Tencent's config: block 16, sliding window 8192, reads target layers 1/16/31/46/61; architectures is DSpineDraftModel, matching Tencent's naming (both names load)
LICENSE, NOTICE Apache-2.0

License and credits

Apache-2.0. Drafter by Tencent Hunyuan, for Qwen3.8-27B by Alibaba's Qwen team. MLX runtime built on Tencent DSpine and mlx-lm. Not affiliated with Tencent, Alibaba/Qwen or Apple.

Made by NetProjects.

Downloads last month
2,569
Safetensors
Model size
2B params
Tensor type
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

Quantized

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for netprojects/Qwen3.8-27B-DSpine-b16-MLX

Base model

Qwen/Qwen3.8-27B
Quantized
(6)
this model

Collection including netprojects/Qwen3.8-27B-DSpine-b16-MLX