--- license: cc-by-nc-4.0 library_name: gguf base_model: CohereLabs/North-Mini-Code-1.0 base_model_relation: quantized model_name: North-Mini-Code-Cerebellum-GGUF model_creator: CohereLabs model_type: cohere2moe quantized_by: deucebucket model-index: - name: North-Mini-Code-Cerebellum-GGUF results: - task: name: Text Generation type: text-generation dataset: name: HumanEval (pass@1) type: openai_humaneval split: test metrics: - name: pass@1 type: pass@1 value: 0.848 source: name: Local run (RTX 3090, llama.cpp, reasoning-aware extraction; thinking-off) url: https://huggingface.co/deucebucket/North-Mini-Code-Cerebellum-GGUF/tree/main/benchmark_results - task: name: Text Generation type: text-generation dataset: name: HumanEval+ (pass@1) type: evalplus/humanevalplus split: test metrics: - name: pass@1 type: pass@1 value: 0.793 source: name: Local run (RTX 3090, llama.cpp, reasoning-aware extraction; thinking-off) url: https://huggingface.co/deucebucket/North-Mini-Code-Cerebellum-GGUF/tree/main/benchmark_results - task: name: Text Generation type: text-generation dataset: name: AI2 Reasoning Challenge type: ai2_arc config: ARC-Challenge split: test metrics: - name: normalized accuracy type: acc_norm value: 0.9224 source: name: Local benchmark run (RTX 3090, llama.cpp) url: https://huggingface.co/deucebucket/North-Mini-Code-Cerebellum-GGUF/tree/main/benchmark_results - task: name: Text Generation type: text-generation dataset: name: HellaSwag type: hellaswag split: validation metrics: - name: accuracy type: acc value: 0.552 source: name: Local benchmark run (RTX 3090, llama.cpp) url: https://huggingface.co/deucebucket/North-Mini-Code-Cerebellum-GGUF/tree/main/benchmark_results - task: name: Text Generation type: text-generation dataset: name: MMLU-Redux type: cais/mmlu config: all split: test metrics: - name: accuracy type: acc value: 0.780 source: name: Local benchmark run (RTX 3090, llama.cpp) url: https://huggingface.co/deucebucket/North-Mini-Code-Cerebellum-GGUF/tree/main/benchmark_results pipeline_tag: text-generation tags: - GGUF - cohere2moe - cerebellum - imatrix - moe - mixed-precision - code - conversational ---

Cerebellum

# North-Mini-Code 1.0 — Cerebellum GGUF Coding-ablation-guided mixed-precision quantization of [CohereLabs/North-Mini-Code-1.0](https://huggingface.co/CohereLabs/North-Mini-Code-1.0) (`cohere2moe`, 128-expert MoE). | Variant | File | Size | |---------|------|------| | **Cerebellum v1** | `North-Mini-Code-Cerebellum-v1.gguf` | **13.57 GB** | Cerebellum measures, **per tensor group, what actually breaks the model's coding ability** when crushed to Q2_K — using real HumanEval pass@1 deltas, not perplexity (perplexity is blind to code generation). It then writes one GGUF that protects the groups that matter and crushes the rest. On North, the per-group HumanEval ablation found exactly **one** coding-critical group — the routed down-projection experts — so v1 keeps those at Q4_K_M and crushes everything else to Q2_K. ## Benchmarks Measured directly on this GGUF with llama.cpp (current master). `--parallel 1`, temperature 0. | Benchmark | North-Mini-Code-Cerebellum-v1 (13.57 GB) | |-----------|:---:| | **HumanEval base** (thinking on / off) | **86.6% / 84.8%** | | **HumanEval+** (thinking on / off) | **82.9% / 79.3%** | | **ARC-Challenge** (1172) | **92.2%** | | **MMLU-Redux** (250) | **78.0%** | | HellaSwag (250) | 55.2% | ### Important: measuring HumanEval correctly North-Mini-Code is a **reasoning-native** model — it emits its full chain-of-thought in the response *content* (no `` delimiter for llama.cpp to split on). A naive EvalPlus/HumanEval harness extracts the reasoning prose instead of the final code and **massively under-scores the model** (~51% on a stock extractor). The numbers above use a reasoning-aware extractor that recovers the model's actual final code block before execution; on inspection, the remaining failures are genuine logic errors plus a small number of reasoning-overflow cases, not extraction noise. If you bench this model, strip the reasoning to the final code block, or disable thinking (below). Per the project's standing rule, comparisons are to the base model and same-size baselines only. ## v1 Allocation Built from the complete per-group HumanEval coding ablation (baseline = uniform Q4_K_M, 8 groups, real pass@1 deltas): | Group | Precision | Why | |-------|-----------|-----| | `ffn_down_exps` (routed) | **Q4_K_M** | The one coding-critical group — Q2_K here dropped HumanEval ~37 pts. Protected. | | `ffn_(gate\|up)_exps` (routed) | Q2_K | Crushing it *improved* coding; ~4.5 GB of the savings | | `attn_q` / `attn_k` / `attn_v` / `attn_output` | Q2_K | Free or beneficial at Q2_K per the ablation | | `token_embd` (tied) | Q2_K | Free / beneficial | | `blk.0` dense ffn | Q2_K | Free | Norms protected (default). Base type Q4_K_M; only the groups above are overridden. ## Requirements This is a `cohere2moe` model — it needs a **llama.cpp build with cohere2-MoE support** (merged to master, PR #24260 / commit `4988f6e`). The GGUF uses the `tiny_aya` pre-tokenizer; older/PR-head builds that expect `cohere2moe` will fail to load. Build from current `ggml-org/llama.cpp` master. ## Usage ```bash # default (reasoning on — the model performs best with thinking enabled) llama-server --model North-Mini-Code-Cerebellum-v1.gguf -ngl 99 -c 8192 --jinja # no-thinking (faster, clean final output) llama-server --model North-Mini-Code-Cerebellum-v1.gguf -ngl 99 -c 8192 --jinja \ --reasoning off --reasoning-budget 0 # (also pass chat_template_kwargs {"enable_thinking": false} in the request) ``` Fits a 16 GB card. The model reasons in the content channel; for agent loops, pass the reasoning content forward between turns (per the base model's guidance). ## Evidence Per-benchmark detail and the coding-ablation results are in `benchmark_results/`.