scragnog/YuE2-GGUF

YuE2 GGUF

GGUF conversions of m-a-p/YuE2-3B and m-a-p/YuE2-Vae for HOT-Step CPP, a local C++/GGML music generation engine. YuE2 is an autoregressive-planner + flow-matching music model from the M-A-P team; these files let it run through HOT-Step's GGML inference stack instead of the original PyTorch checkpoint.

Attribution

All model weights here are derived from the original YuE2 release. Credit and thanks to the YuE2 authors (M-A-P) for the underlying research and checkpoints:

• LM: https://huggingface.co/m-a-p/YuE2-3B
• VAE: https://huggingface.co/m-a-p/YuE2-Vae

These are unofficial GGUF conversions, not affiliated with or endorsed by the YuE2 authors. Nothing here changes the model's behavior beyond the numeric effects of quantization documented below.

Licence

The YuE2 checkpoint weights (including these GGUF conversions) are licensed under CC BY-NC 4.0 (Creative Commons Attribution-NonCommercial 4.0 International): https://creativecommons.org/licenses/by-nc/4.0/

This means: attribution required, non-commercial use only. See LICENSE for the full text and attribution scope. For a commercial licence, contact the upstream YuE2 team directly: gezhang@umich.edu.

Third-party code notices (the Oobleck VAE / SnakeBeta implementation this port is derived from) are in THIRD_PARTY_NOTICES.md.

Source revisions

These files were converted from the following pinned upstream commits:

Component	Source repo	Revision
YuE2-3B (LM)	m-a-p/YuE2-3B	1a96eca688d6ae5d7f0feb88573fec89920fcd19
YuE2-Vae (standard VAE)	m-a-p/YuE2-Vae	95535e72a97bc0f09b8ada125d26b4009428c0e8
YuE2-Vae (legacy VAE)	m-a-p/YuE2-Vae	b54118f0fc462f08999d1ec07e88817f4ee3f770

Files

The VAE stays F32-only by design — see "A note on the VAE" below.

File	Size	Notes
yue2-lm-bf16.gguf	7.27 GB	Reference / source dtype. Near-lossless vs the original safetensors.
yue2-lm-q8_0.gguf	3.92 GB	Recommended default. Near-lossless (argmax 100%, NAR rel-L2 0.019 vs bf16's 0.013).
yue2-lm-Q6_K.gguf	3.05 GB	Plain quant, no imatrix.
yue2-lm-Q5_K_M.gguf	2.73 GB	Plain quant, no imatrix.
yue2-lm-Q4_K_M-imat.gguf	2.43 GB	Imatrix-guided.
yue2-lm-Q4_K_S-imat.gguf	2.34 GB	Imatrix-guided.
yue2-lm-IQ4_XS-imat.gguf	2.24 GB	Imatrix-guided i-quant.
yue2-lm-Q3_K_M-imat.gguf	2.06 GB	Imatrix-guided; below this size, quality loss becomes audible-territory.
yue2-lm-Q3_K_S-imat.gguf	1.95 GB	Imatrix-guided.
yue2-lm-Q2_K-imat.gguf	1.69 GB	Imatrix-guided. Imatrix matters most here — halves the error of the plain Q2_K quant.
yue2-lm-IQ3_XXS-imat.gguf	1.73 GB	Imatrix-guided i-quant.
yue2-lm-IQ2_XS-imat.gguf	1.37 GB	Imatrix-guided i-quant. Smallest usable file; expect real quality loss.
yue2-lm-NVFP4.gguf	2.51 GB	FP4 format. Scores below the k-quants at the same size (see quality summary) — kept for completeness/testing, not recommended over Q4_K_M or Q3_K_M.
yue2-lm-MXFP4.gguf	2.42 GB	FP4 format. Same caveat as NVFP4, and the weaker of the two FP4 formats measured.
yue2-lm-imatrix.gguf	4.2 MB	The importance-matrix file itself (12 calibration renders across 4 genres/languages, cot off and full). Only needed if you want to build your own quant with engine/tools/quantize.cpp.
yue2-vae-standard-f32.gguf	530 MB	Standard VAE. F32, not quantized — do not requantize.
yue2-vae-legacy-f32.gguf	530 MB	Legacy VAE variant. F32, not quantized — do not requantize.
tokenizer/	~4.8 MB	BPE vocab/merges + pretokenizer/special-token config the LM needs at load time.

Superseded plain quants (Q4_K_M, Q3_K_M, Q2_K without imatrix) were built during development but are not published here — their -imat counterparts measure strictly better or comparably at the same file size, so there's no reason to ship both.

How the quants were made

Conversion: engine/tools/convert-yue2.py (safetensors → GGUF, BF16 source). Quantization: engine/tools/quantize.cpp, HOT-Step's shared GGML quantizer (the same tool used for the ACE-Step and MiniMax-Music3 GGUFs in this repo).

The -imat files were built with --imatrix yue2-lm-imatrix.gguf, an importance matrix collected from 12 real /yue2/synth generations spanning 4 genres/languages (English indie folk, Mandarin pop ballad, hip-hop, metal) with cot both off and on, 259M+ calibration rows across 397 of 399 matmul-reachable tensors. Two tensors (token_embd.weight, latent_pos_embed.weight) are read via ggml_get_rows rather than ggml_mul_mat and so fall back to round-to-nearest regardless of imatrix presence — latent_pos_embed.weight (and three other 2-D tensor families: vae2llm, llm2vae, time_embd.{0,1}) are excluded from quantization entirely and always ship as float.

Quality summary

Measured with teacher-forced fixtures against the bf16 reference — not an ear/listening evaluation. From docs/plans/yue2/07-quant-ladder.md in the HOT-Step CPP repo:

• q8_0 is near-lossless: 100% argmax agreement with bf16, mean relative logit error barely moves, NAR (flow-matching) output relative-L2 error of 0.019 vs bf16's own 0.013. If you want any quantization at all, this is the safe default.
• Imatrix is strongly recommended below Q5. At Q4_K_M it's a modest, consistent win across every metric. At Q2_K the difference is large — the imatrix version roughly halves both mean logit error and NAR relative-L2 error versus the plain (non-imatrix) Q2_K quant. Below Q5_K_M, prefer the -imat file.
• Argmax agreement never drops below 100%, even at Q2_K — degradation shows up in logit magnitude and in the NAR/flow-matching path's relative-L2 error, not in which token a greedy decode would pick. This matters if anything downstream reads logits directly (temperature/top-p sampling, CFG blending) rather than just the argmax.
• NVFP4 and MXFP4 both score below the k-quants at a matched file size. Both land in the same 2.2–2.5 GB band as Q4_K_M, but score worse than even Q3_K_M (a smaller file) on every accuracy metric measured. MXFP4 is the weaker of the two FP4 formats. Neither is recommended over the K-quant ladder at this model's weight distribution — treat this as one data point, not a general FP4 verdict.
• The NAR (flow-matching) path degrades faster than the AR path at the same nominal quant level — it accumulates error across 32 ODE solver steps, so a quant level that looks fine on AR-only argmax checks can still produce audibly worse song output.
• None of this was validated by ear. Treat the numbers as a fidelity ranking, not a listening verdict — especially below Q4_K_M.

A note on the VAE

Both VAE files ship as F32 only. HOT-Step's quantizer refuses to touch anything under the yue2-vae architecture — decode quality is the entire point of that file, and unlike the LM, there's no quantized VAE tier offered or planned.