license: cc-by-nc-4.0
library_name: gguf
base_model: CohereLabs/North-Mini-Code-1.0
base_model_relation: quantized
model_name: North-Mini-Code-Cerebellum-GGUF
model_creator: CohereLabs
model_type: cohere2moe
quantized_by: deucebucket
model-index:
- name: North-Mini-Code-Cerebellum-GGUF
results:
- task:
name: Text Generation
type: text-generation
dataset:
name: HumanEval (pass@1)
type: openai_humaneval
split: test
metrics:
- name: pass@1
type: pass@1
value: 0.848
source:
name: >-
Local run (RTX 3090, llama.cpp, reasoning-aware extraction;
thinking-off)
url: >-
https://huggingface.co/deucebucket/North-Mini-Code-Cerebellum-GGUF/tree/main/benchmark_results
- task:
name: Text Generation
type: text-generation
dataset:
name: HumanEval+ (pass@1)
type: evalplus/humanevalplus
split: test
metrics:
- name: pass@1
type: pass@1
value: 0.793
source:
name: >-
Local run (RTX 3090, llama.cpp, reasoning-aware extraction;
thinking-off)
url: >-
https://huggingface.co/deucebucket/North-Mini-Code-Cerebellum-GGUF/tree/main/benchmark_results
- task:
name: Text Generation
type: text-generation
dataset:
name: AI2 Reasoning Challenge
type: ai2_arc
config: ARC-Challenge
split: test
metrics:
- name: normalized accuracy
type: acc_norm
value: 0.9224
source:
name: Local benchmark run (RTX 3090, llama.cpp)
url: >-
https://huggingface.co/deucebucket/North-Mini-Code-Cerebellum-GGUF/tree/main/benchmark_results
- task:
name: Text Generation
type: text-generation
dataset:
name: HellaSwag
type: hellaswag
split: validation
metrics:
- name: accuracy
type: acc
value: 0.552
source:
name: Local benchmark run (RTX 3090, llama.cpp)
url: >-
https://huggingface.co/deucebucket/North-Mini-Code-Cerebellum-GGUF/tree/main/benchmark_results
- task:
name: Text Generation
type: text-generation
dataset:
name: MMLU-Redux
type: cais/mmlu
config: all
split: test
metrics:
- name: accuracy
type: acc
value: 0.78
source:
name: Local benchmark run (RTX 3090, llama.cpp)
url: >-
https://huggingface.co/deucebucket/North-Mini-Code-Cerebellum-GGUF/tree/main/benchmark_results
pipeline_tag: text-generation
tags:
- GGUF
- cohere2moe
- cerebellum
- imatrix
- moe
- mixed-precision
- code
- conversational
North-Mini-Code 1.0 — Cerebellum GGUF
Coding-ablation-guided mixed-precision quantization of CohereLabs/North-Mini-Code-1.0 (cohere2moe, 128-expert MoE).
| Variant | File | Size |
|---|---|---|
| Cerebellum v1 | North-Mini-Code-Cerebellum-v1.gguf |
13.57 GB |
Cerebellum measures, per tensor group, what actually breaks the model's coding ability when crushed to Q2_K — using real HumanEval pass@1 deltas, not perplexity (perplexity is blind to code generation). It then writes one GGUF that protects the groups that matter and crushes the rest. On North, the per-group HumanEval ablation found exactly one coding-critical group — the routed down-projection experts — so v1 keeps those at Q4_K_M and crushes everything else to Q2_K.
Benchmarks
Measured directly on this GGUF with llama.cpp (current master). --parallel 1, temperature 0.
| Benchmark | North-Mini-Code-Cerebellum-v1 (13.57 GB) |
|---|---|
| HumanEval base (thinking on / off) | 86.6% / 84.8% |
| HumanEval+ (thinking on / off) | 82.9% / 79.3% |
| ARC-Challenge (1172) | 92.2% |
| MMLU-Redux (250) | 78.0% |
| HellaSwag (250) | 55.2% |
Important: measuring HumanEval correctly
North-Mini-Code is a reasoning-native model — it emits its full chain-of-thought in the response content (no <think> delimiter for llama.cpp to split on). A naive EvalPlus/HumanEval harness extracts the reasoning prose instead of the final code and massively under-scores the model (~51% on a stock extractor). The numbers above use a reasoning-aware extractor that recovers the model's actual final code block before execution; on inspection, the remaining failures are genuine logic errors plus a small number of reasoning-overflow cases, not extraction noise. If you bench this model, strip the reasoning to the final code block, or disable thinking (below).
Per the project's standing rule, comparisons are to the base model and same-size baselines only.
v1 Allocation
Built from the complete per-group HumanEval coding ablation (baseline = uniform Q4_K_M, 8 groups, real pass@1 deltas):
| Group | Precision | Why |
|---|---|---|
ffn_down_exps (routed) |
Q4_K_M | The one coding-critical group — Q2_K here dropped HumanEval ~37 pts. Protected. |
ffn_(gate|up)_exps (routed) |
Q2_K | Crushing it improved coding; ~4.5 GB of the savings |
attn_q / attn_k / attn_v / attn_output |
Q2_K | Free or beneficial at Q2_K per the ablation |
token_embd (tied) |
Q2_K | Free / beneficial |
blk.0 dense ffn |
Q2_K | Free |
Norms protected (default). Base type Q4_K_M; only the groups above are overridden.
Requirements
This is a cohere2moe model — it needs a llama.cpp build with cohere2-MoE support (merged to master, PR #24260 / commit 4988f6e). The GGUF uses the tiny_aya pre-tokenizer; older/PR-head builds that expect cohere2moe will fail to load. Build from current ggml-org/llama.cpp master.
Usage
# default (reasoning on — the model performs best with thinking enabled)
llama-server --model North-Mini-Code-Cerebellum-v1.gguf -ngl 99 -c 8192 --jinja
# no-thinking (faster, clean final output)
llama-server --model North-Mini-Code-Cerebellum-v1.gguf -ngl 99 -c 8192 --jinja \
--reasoning off --reasoning-budget 0
# (also pass chat_template_kwargs {"enable_thinking": false} in the request)
Fits a 16 GB card. The model reasons in the content channel; for agent loops, pass the reasoning content forward between turns (per the base model's guidance).
Evidence
Per-benchmark detail and the coding-ablation results are in benchmark_results/.