ukuwzi/Qweble-Sol-4B-GGUF

Qweble-Sol 4B

Qweble-Sol is a fine-tuned derivative of Qwen3.5-4B, focused on systems administration, programming, troubleshooting, DevOps, and technical assistance.

• Base model: Qwen3.5-4B
• Base model developer: Alibaba / Qwen Team
• Fine-tuned and packaged by: ukuwzi
• Parameters: ~4.21B
• Primary use: Technical assistance, coding, systems administration, debugging, networking, and DevOps
• Native fine-tuning/runtime: MLX / mlx-lm
• Available formats: MLX 4-bit and GGUF Q4_K_M
• GGUF runtimes tested: llama.cpp and Ollama

Qweble-Sol is an independently fine-tuned derivative model. It is not an official Qwen or Alibaba model.

---

Overview

Qweble-Sol is intended to be a compact local technical assistant capable of helping with tasks such as:

• Linux and macOS administration
• Shell scripting
• Python, Go, Rust, and JavaScript
• Debugging and code review
• Docker and container workflows
• Networking and troubleshooting
• Server configuration
• Automation
• CI/CD and DevOps
• Security best practices
• General technical questions

The goal of the fine-tune is to preserve the general reasoning and conversational abilities of Qwen3.5-4B while specializing its behavior toward practical technical and systems-oriented tasks.

Qweble-Sol supports Qwen3.5's reasoning/thinking behavior.

---

Model Lineage

Qwen3.5-4B
    │
    ▼
MLX fine-tuning
    │
    ▼
Qweble-Sol 16-bit MLX
    │
    ├──────────────► Native MLX inference
    │
    ▼
HF-compatible weight export
    │
    ▼
F16 GGUF
    │
    ▼
Q4_K_M GGUF
    │
    ├──────────────► llama.cpp
    │
    └──────────────► Ollama

The original Qwen3.5-4B model was developed by the Qwen Team at Alibaba.

Qweble-Sol was fine-tuned, adapted, converted, and packaged by ukuwzi.

---

Available Formats

MLX 16-bit

The MLX release is the native fused version of Qweble-Sol and is recommended for Apple Silicon users who want maximum model fidelity.

Typical directory contents:

config.json
model-00001-of-00002.safetensors
model-00002-of-00002.safetensors
model.safetensors.index.json
tokenizer.json
tokenizer_config.json
...

Approximate model size:

~7.8 GB

Run with MLX

Install MLX-LM:

pip install -U mlx-lm

Then run:

mlx_lm.chat --model /path/to/Qweble-Sol-4B-MLX

For generation:

mlx_lm.generate \
  --model /path/to/Qweble-Sol-4B-MLX \
  --prompt "Explain Docker in one paragraph." \
  --max-tokens 256

---

GGUF Q4_K_M

The GGUF release is intended for:

• Ollama
• llama.cpp
• Other GGUF-compatible runtimes

Quantization:

Q4_K_M

Model size:

2.51 GiB

This version offers substantially reduced storage and memory requirements compared with the 16-bit MLX model.

---

llama.cpp Usage

Run Qweble-Sol directly with llama.cpp:

./llama-cli \
  -m Qweble-Sol-4B-Q4_K_M.gguf

With a prompt:

./llama-cli \
  -m Qweble-Sol-4B-Q4_K_M.gguf \
  -p "Explain the difference between a Docker image and a container."

A moderate context length such as 8192 tokens is recommended as a starting point on machines with limited unified memory:

./llama-cli \
  -m Qweble-Sol-4B-Q4_K_M.gguf \
  -c 8192

---

Ollama

Qweble-Sol is available directly from the Ollama registry.

Run

ollama run ukuwzi/qweble-sol:4b-q4_k_m

Ollama will automatically download the model on first use. You can view the Ollama model page for more details.

API

Qweble-Sol can also be used through the local Ollama API:

curl http://localhost:11434/api/chat \
  -d '{
    "model": "ukuwzi/qweble-sol:4b-q4_k_m",
    "messages": [
      {
        "role": "user",
        "content": "Explain how Docker containers work."
      }
    ]
  }'

---

Performance

Test System

The following results were measured locally on:

Component	Configuration
Hardware	Apple M4 Pro
Unified memory	24 GB
OS	macOS
GGUF runtime	llama.cpp
GGUF backend	Metal + BLAS
GGUF quantization	Q4_K_M
MLX runtime	mlx-lm
llama.cpp build	e613ef2c8 / b11056

These numbers should be treated as hardware-specific measurements rather than universal performance estimates.

---

GGUF / llama.cpp Performance

Measured with llama-bench, using five repetitions per test.

Test	Throughput
Prompt processing — 512 tokens	776.14 ± 0.96 tok/s
Prompt processing — 1024 tokens	773.16 ± 1.88 tok/s
Prompt processing — 2048 tokens	766.76 ± 0.15 tok/s
Generation — 128 tokens	65.75 ± 0.12 tok/s
Generation — 256 tokens	65.63 ± 0.03 tok/s

Model information reported by llama-bench:

Parameters: 4.21B
GGUF size: 2.51 GiB
Quantization: Q4_K_M
Backend: Metal + BLAS
Threads: 10

Summary

On the tested M4 Pro:

Prompt processing: ~767–776 tokens/second
Token generation:  ~65.6 tokens/second

---

Native MLX Performance

Measured using the fused 16-bit MLX version:

Prompt tokens: 19
Generated tokens: 256

Results:

Metric	Result
Prompt processing	83.92 tok/s
Generation	27.84 tok/s
Peak memory	8.52 GB
Precision	16-bit

Important Benchmark Note

The MLX and llama.cpp results use different benchmark methodologies.

llama-bench isolates model inference throughput, while the MLX result comes from mlx_lm.generate.

The numbers therefore should not be interpreted as a direct apples-to-apples comparison between MLX and GGUF.

---

Benchmark Summary

Runtime	Format	Model Size	Prompt	Generation	Peak Memory
MLX / mlx-lm	16-bit	~7.8 GB	83.92 tok/s	27.84 tok/s	8.52 GB
llama.cpp	Q4_K_M	2.51 GiB	~767–776 tok/s*	~65.6 tok/s*	Not yet measured

*Measured with llama-bench; benchmark methodologies differ between runtimes.

---

Intelligence / Quality Benchmarks

Formal quality benchmarks are still being evaluated.

Planned evaluations include:

Benchmark	Focus	Qweble-Sol
MMLU-Pro	General knowledge + reasoning	TBD
AIME	Mathematical reasoning	TBD
MBPP / MBPP+	Python programming	TBD
LiveCodeBench	Code generation	TBD
Sysadmin Eval	Systems administration	TBD

Future results should ideally be reported alongside the original Qwen3.5-4B under equivalent inference settings.

This section will be updated as evaluations are completed.

---

Qweble-Sol Sysadmin Evaluation

Because Qweble-Sol is specifically oriented toward technical and systems-administration work, general academic benchmarks alone may not accurately represent its intended strengths.

A dedicated evaluation is planned across categories such as:

Linux troubleshooting
macOS administration
Networking
Shell scripting
Python debugging
Docker
Server administration
DevOps
Automation
Security fundamentals

Results and evaluation prompts may be published separately for reproducibility.

---

Recommended Generation Settings

A reasonable starting configuration is:

temperature: 0.6
top_p: 0.95
top_k: 20
context: 8192

These are recommendations rather than strict requirements.

For deterministic benchmarking, use a temperature of 0 where supported.

---

Thinking / Reasoning

Qweble-Sol retains Qwen3.5's thinking-style response behavior.

Depending on the runtime, reasoning may appear as:

Thinking...
...
done thinking.

or through <think> blocks.

Different runtimes may display or suppress this reasoning differently.

---

Conversion Notes

Important Qwen3.5 + MLX Compatibility Note

Qweble-Sol was fine-tuned and fused using MLX.

A direct conversion of the MLX-fused Qwen3.5 checkpoint using:

convert_hf_to_gguf.py

produced a GGUF that loaded successfully but generated corrupted output.

The MLX model itself remained fully functional.

The issue resulted from differences between the MLX-native representation of some Qwen3.5 tensors and the canonical Hugging Face representation expected by the GGUF conversion pipeline.

To produce the working GGUF release:

1. The original MLX model was preserved unchanged.
2. A separate copy was exported back into HF-compatible tensor semantics.
3. The corrected copy was converted to F16 GGUF.
4. The F16 GGUF was validated with llama.cpp.
5. The model was quantized to Q4_K_M.
6. The Q4_K_M release was validated with both llama.cpp and Ollama.

The native MLX checkpoint remains the canonical MLX version.

This distinction is important for anyone attempting to reproduce a Qwen3.5 MLX → GGUF conversion.

---

Known Working Configurations

Qweble-Sol has been locally validated with:

MLX

Apple Silicon
mlx-lm
16-bit fused checkpoint

llama.cpp

Q4_K_M GGUF
Metal backend
Apple M4 Pro

Ollama

Q4_K_M GGUF
8192-token configured context
Custom Qweble-Sol system prompt

---

Intended Uses

Qweble-Sol is primarily intended for:

• Local technical assistance
• Programming help
• Code explanation
• Debugging
• Shell scripting
• Systems troubleshooting
• Linux/macOS administration
• Docker help
• Networking concepts
• DevOps workflows
• Automation
• Educational technical assistance

It can also perform ordinary general-purpose conversational and reasoning tasks inherited from its base model.

---

Limitations

Qweble-Sol is a relatively small ~4B-parameter model.

It may:

• Produce incorrect technical information
• Invent commands, flags, APIs, packages, or configuration options
• Misinterpret ambiguous infrastructure problems
• Produce code that requires additional testing
• Give outdated answers about rapidly changing software
• Make mistakes in long or complicated reasoning chains
• Incorrectly describe its own training history or provenance without an explicit system prompt
• Behave differently across MLX, llama.cpp, and Ollama because of runtime and prompt-template differences

Always verify commands before running them against important systems.

Extra care should be taken with:

rm
dd
disk utilities
partitioning
firewall changes
permissions
production databases
deployment commands
system configuration

Back up important data before performing destructive operations.

---

Training Details

Qweble-Sol was fine-tuned using MLX / mlx-lm.

Base

Qwen/Qwen3.5-4B

Fine-tuning

Framework: mlx-lm
Method: Adapter/LoRA fine-tuning followed by fusion
Fine-tuner: ukuwzi

Additional dataset and hyperparameter details can be added here if/when they are publicly released.

---

Architecture

Qweble-Sol retains the underlying Qwen3.5-4B architecture.

High-level properties of the underlying language model include:

Parameters: ~4B
Hidden dimension: 2560
Transformer/hybrid blocks: 32
Feed-forward dimension: 9216
Hybrid linear-attention + full-attention architecture

The GGUF build reports approximately 4.21B parameters.

---

Model Provenance

To avoid ambiguity:

Item	Developer
Qwen3.5 architecture / base model	Alibaba / Qwen Team
Qwen3.5-4B base weights	Alibaba / Qwen Team
Qweble-Sol fine-tune	ukuwzi
Qweble-Sol MLX packaging	ukuwzi
Qweble-Sol GGUF conversion	ukuwzi
Qweble-Sol Ollama packaging	ukuwzi

Qweble-Sol is not an official Alibaba or Qwen release.

---

License

Qweble-Sol is derived from Qwen3.5-4B, which is distributed under the Apache License 2.0.

Users should review the upstream Qwen3.5 license and any applicable terms before redistribution or commercial use.

See the upstream model:

Qwen/Qwen3.5-4B

---

Acknowledgements

Qweble-Sol would not exist without the work of:

• Qwen Team / Alibaba — Qwen3.5-4B
• MLX / mlx-lm contributors — Apple Silicon fine-tuning and inference
• llama.cpp contributors — GGUF conversion and local inference
• Ollama contributors — simple local model deployment

---

Citation

If you use Qweble-Sol in a project, benchmark, or derivative work, you can reference the Hugging Face repository.

A formal citation entry may be added in a future release.

Please also cite the original Qwen3.5 work when appropriate.

---

Status

MLX 16-bit       ✅ Working
GGUF F16         ✅ Working
GGUF Q4_K_M      ✅ Working
llama.cpp        ✅ Tested
Ollama           ✅ Tested
MMLU-Pro         ⏳ Pending
Coding benchmarks ⏳ Pending
Sysadmin eval    ⏳ Pending

---

Disclaimer

Qweble-Sol can generate incorrect or unsafe technical instructions. Generated commands and code should be reviewed before execution, especially on production systems or systems containing important data.

This project is an independent fine-tune and is not affiliated with or endorsed by Alibaba, Qwen, Apple, llama.cpp, or Ollama.