How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ambrosfitz/mistral-nemo-humanities-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf ambrosfitz/mistral-nemo-humanities-gguf:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ambrosfitz/mistral-nemo-humanities-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf ambrosfitz/mistral-nemo-humanities-gguf:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf ambrosfitz/mistral-nemo-humanities-gguf:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf ambrosfitz/mistral-nemo-humanities-gguf:Q4_K_M
Use Docker
docker model run hf.co/ambrosfitz/mistral-nemo-humanities-gguf:Q4_K_M
Quick Links

Mistral Nemo 12B - Humanities Distilled (GGUF)

This model is a distilled version of Mistral Nemo 12B, fine-tuned on humanities content including:

  • MMLU Humanities datasets (history, philosophy, religion, ethics)
  • Synthetic data generated from GPT-OSS-120B

Training Details

  • Base Model: mistralai/Mistral-Nemo-Instruct-2407
  • Teacher Model: openai/gpt-oss-120b
  • Training Method: LoRA with 8-bit quantization
  • Dataset: MMLU humanities + synthetic data
  • Specialization: History, philosophy, and humanities understanding

Available Quantizations

  • FP16 (model-f16.gguf): Full precision, ~24GB
  • Q8_0 (model-Q8_0.gguf): 8-bit quantization, ~13GB, high quality
  • Q4_K_M (model-Q4_K_M.gguf): 4-bit quantization, ~7GB, recommended for most users

Usage with llama.cpp

# Download model
huggingface-cli download ambrosfitz/mistral-nemo-gguf model-Q4_K_M.gguf --local-dir ./models

# Run inference
./llama.cpp/main -m ./models/model-Q4_K_M.gguf -p "Question: What was the Renaissance?\n\nAnswer:" -n 256

Usage with Ollama

# Create Modelfile
cat > Modelfile <<EOF
FROM ./model-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.9
EOF

# Create model
ollama create mistral-humanities -f Modelfile

# Run
ollama run mistral-humanities "What was the Renaissance?"

Example Prompts

Question: What were the main causes of World War I?

Answer:

Question: Explain the philosophical ideas of the Enlightenment.

Answer:

Question: Who wrote 'The Republic' and what is it about?

Answer:

Limitations

This model is specialized for humanities topics and may not perform as well on:

  • Technical/scientific questions
  • Mathematics
  • Coding
  • Current events (knowledge cutoff applies)

License

Apache 2.0 (same as base model)

Downloads last month
53
GGUF
Model size
12B params
Architecture
llama
Hardware compatibility
Log In to add your hardware

4-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for ambrosfitz/mistral-nemo-humanities-gguf