darkmaniac7/Dolphin3.0-Llama3.2-3B-MNN

Dolphin 3.0 Llama 3.2 3B (MNN)

Dolphin 3.0 Llama 3.2 3B, the compact steerable fine-tune by Eric Hartford and Cognitive Computations, as a 4-bit MNN bundle for chatting on your phone with TokForge.

Published by TokForge for on-device inference.

What it is	Chat model (fine-tune)
Format	MNN bundle, 4-bit weights (block 128)
Runs on	Android phones (CPU, OpenCL or Vulkan GPU)
Phone class	No catalog class; needs more free RAM than its 2.60 GB of weights
Download	2.60 GB (10 files)
Base model	cognitivecomputations/Dolphin3.0-Llama3.2-3B
License	Llama 3.2 Community License

TokForge

• Website: https://tokforge.ai
• Discord: https://discord.gg/Acv3CBtfVm
• Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
• iOS TestFlight: https://testflight.apple.com/join/jnufjzRr

Runs on-device in the TokForge app.

Use it in TokForge

1. Install TokForge from Google Play, or join the iOS TestFlight.
2. On Android, open Models > Browse, search for Dolphin3.0-Llama3.2-3B-MNN and download it.
3. Load it and start a chat. The model runs on the phone; once it is downloaded, chatting needs no connection.

Collections: TokForge Llama, Mistral and Dolphin (MNN).

Files

File	Size	What it is
config.json	210 B	MNN runtime settings
embeddings_bf16.bin	788 MB	Token embeddings (bf16)
export_args.json	1.12 KB	Export settings used for the conversion
llm.mnn	489 KB	Model graph
llm.mnn.json	983 KB	Model graph as JSON (reference copy)
llm.mnn.weight	1.81 GB	Model weights
llm_config.json	2.89 KB	Model settings and chat template
tokenizer.txt	1.41 MB	Tokenizer
LICENSE	7.71 KB	Llama 3.2 Community License (Meta's text)
NOTICE	116 B	Llama 3.2 notice (Meta)

About the model

Dolphin 3.0 compact, the 3B variant of Eric Hartford's Dolphin series, is a fine-tune of Llama 3.2 3B. It takes the same steerable approach as the 8B: it uses ChatML prompts and follows the system prompt for tone, rules and alignment. It is a fine-tune, not an abliteration. It needs less memory than the 8B, at some cost in answer quality.

Architecture	Llama 3.2 (standard attention, 28 layers, GQA 24Q/8KV)
Parameters	3B (4-bit quantized)
Quantization	W4A16 (4-bit weights, block size 128)
Vocab	128,258 tokens

Recommended settings

Setting	Value
Backend	OpenCL (Qualcomm) / Vulkan (MediaTek) / CPU (fallback)
Precision	Low
Threads	4
Thinking	Off

Set a system prompt: Dolphin follows it for tone and rules, and without one it falls back to a default behavior.

How it was made

Converted using MNN's llmexport pipeline:

python llmexport.py --path cognitivecomputations/Dolphin3.0-Llama3.2-3B --export mnn --quant_bit 4 --quant_block 128

Limitations

• Intended for TokForge / MNN on-device inference on Android
• This is a runtime bundle, not a standard Transformers training checkpoint
• Quantization (Q4) may slightly reduce quality compared to the full-precision original
• This model gives fewer built-in refusals than its base model: use responsibly

License

Dolphin 3.0 Llama 3.2 3B is built on Meta's Llama 3.2 3B. Llama 3.2 is licensed under the Llama 3.2 Community License. Meta's license text is in LICENSE and the required notice is in NOTICE. Built with Llama.

Credits

• Dolphin 3.0 Llama 3.2 3B by Eric Hartford and Cognitive Computations (dphn) (Hugging Face). Used under the Llama 3.2 Community License.
• Llama 3.2 3B by Meta (Hugging Face). Used under the Llama 3.2 Community License.
• Built with Llama. Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved. The license text and this notice are in the LICENSE and NOTICE files in this repo.
• Converted with llmexport from MNN by Alibaba (Apache-2.0).