Instructions to use Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M
Use Docker
docker model run hf.co/Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M
- Ollama
How to use Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF with Ollama:
ollama run hf.co/Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF with Docker Model Runner:
docker model run hf.co/Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M
- Lemonade
How to use Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Ornith-1.5-9B-BuildEng-V9-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Ornith 1.5 9B โ BuildEng V9 (GGUF)
Engineering-focused fine-tune by Irfan Uruchi.
Base model: ornith-ai/Ornith-1.5-9B
Published formats: Q4_K_M, Q5_K_M, Q6_K
Intended interface: Text in, text out
BuildEng V9 adapts Ornith 1.5 9B for technical discussions of building systems and construction engineering: structural load paths, reinforced-concrete distress, construction changes, temporary works, engineering assessment, and the identification of missing assumptions or required checks.
This is a technical discussion and drafting assistant, not a structural-analysis solver, a code-compliance checker, or an independently validated engineering decision system.
Available quantizations
| File | Format | Approximate download size | When to choose it |
|---|---|---|---|
Ornith-1.5-9B-BuildEng-V9-Q4_K_M.gguf |
Q4_K_M | 5.68 GB | Smallest download; recommended starting point for limited memory. |
Ornith-1.5-9B-BuildEng-V9-Q5_K_M.gguf |
Q5_K_M | 6.49 GB | A middle ground between weight precision and storage. |
Ornith-1.5-9B-BuildEng-V9-Q6_K.gguf |
Q6_K | 7.36 GB | Highest weight precision among these GGUF releases. |
Sizes are approximate decimal gigabytes. No comparative GGUF quality, throughput, latency, or peak-memory benchmarks have been established. Required runtime memory exceeds the model file size and depends on the backend, context length, and KV cache.
Usage
Use a recent llama.cpp build with support for this Qwen3.5-derived architecture. The GGUF files can also be loaded in LM Studio or Jan when their bundled inference runtimes support the architecture; these applications have not been independently tested with this release.
Download Q4_K_M with the Hugging Face CLI:
hf download Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF \
Ornith-1.5-9B-BuildEng-V9-Q4_K_M.gguf \
--local-dir ./buildeng-v9
Run a single-turn text-generation check:
llama-cli \
-m ./buildeng-v9/Ornith-1.5-9B-BuildEng-V9-Q4_K_M.gguf \
--jinja \
--reasoning off \
--conversation \
--single-turn \
-p "Explain the load path of a transfer beam and identify the checks required before changing its support conditions."
--reasoning off requests non-thinking responses; the exact behavior depends on the model chat template and llama.cpp version. Check llama-cli --help if your build does not recognize an option.
These GGUF files are intended for text generation. Image, audio, and other multimodal inputs have not been validated for this release.
Training and conversion provenance
BuildEng V9 is derived from Ornith 1.5 9B, using domain-focused fine-tuning associated with the BuildEng dataset.
The GGUF conversion followed these steps:
- Adapter fusion: Fuse the BuildEng V9 LoRA adapter into an Ornith 1.5 9B MLX 8-bit base.
- Dequantization: Export the fused quantized representation as floating-point tensors. Dequantization does not recover the original BF16 precision.
- Tensor canonicalization: Reverse MLX-specific Qwen3.5 normalization offsets and restore the expected convolution tensor layout for conversion.
- GGUF export: Convert the canonicalized checkpoint to F16 GGUF using llama.cpp tooling.
- Quantization: Create Q4_K_M, Q5_K_M, and Q6_K GGUF distributions.
The GGUF files are not direct repackagings of the published MLX 4-bit checkpoint. They use the V9 adapter fused into an MLX 8-bit base, and numerical equivalence between these release paths has not been established.
Related MLX releases
- BuildEng V9 โ MLX 3-bit
- BuildEng V9 โ MLX 4-bit
- BuildEng V9 โ MLX 6-bit
- BuildEng V9 โ MLX 8-bit
Validation status
Completed:
- Functional text-generation smoke tests for Q4_K_M, Q5_K_M, and Q6_K in the release workflow.
- Hugging Face upload checks for all three GGUF weight files.
Not established by these checks:
- Accuracy on structural-engineering benchmarks or actual engineering case assessments.
- Behavioral equivalence to upstream Ornith or to the MLX releases.
- GGUF inference throughput, latency, or peak-memory characteristics.
- Independent functional testing in desktop GUI applications.
- Engineering-code compliance, calculation reliability, or suitability for safety-critical decisions.
For context, the separate MLX 4-bit release measured 29.46 generated tokens/s in one inference run on an Apple M6 Mac mini. This is not a GGUF benchmark and does not predict performance across GGUF runtimes.
Limitations and responsible use
BuildEng V9 can help organize technical discussions, draft preliminary checklists, and identify missing information. However, it can produce incorrect calculations, omit important load cases, invent code provisions, misinterpret units or material properties, or draw unjustified conclusions from incomplete inputs.
For actual building work, independently verify assumptions, drawings, measurements, loading conditions, applicable codes and standards, and calculations. Decisions affecting structural safety require review by a qualified engineer.
Do not use this model alone to approve structural modifications, diagnose stability, direct temporary works, or authorize site activity.
License and attribution
- BuildEng V9 fine-tuning and GGUF release: Irfan Uruchi.
- Base model: Ornith AI โ Ornith 1.5 9B, distributed under MIT.
- Underlying Qwen components: Subject to relevant Apache-2.0 notices.
See the repository's LICENSE and LICENSE-QWEN-APACHE-2.0 files for the relevant license texts. Upstream credit belongs to Ornith AI and the Qwen team. BuildEng V9 is an independent fine-tuned derivative and is not an official upstream release.
- Downloads last month
- 112
4-bit
5-bit
6-bit
Model tree for Irfanuruchi/Ornith-1.5-9B-BuildEng-V9-GGUF
Base model
ornith-ai/Ornith-1.5-9B