Image-Text-to-Text
MLX
Safetensors
mage_vl
vision-language-model
video-understanding
mage-vl
conversational
custom_code
8-bit precision
Instructions to use mlx-community/Mage-VL-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/Mage-VL-8bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("mlx-community/Mage-VL-8bit") config = load_config("mlx-community/Mage-VL-8bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Correct the activation figure: 14 GB is the 8192-token budget, not the 4096 default
Browse files
README.md
CHANGED
|
@@ -31,9 +31,27 @@ Gated against the bf16 checkpoint and the PyTorch reference on the model's own
|
|
| 31 |
| int8 vs bf16 MLX | 48/48 tokens identical |
|
| 32 |
| final-position logits vs reference | cos 0.9982 (bf16 baseline: 0.9996) |
|
| 33 |
|
| 34 |
-
Measured on an M5 Max (macOS 27)
|
| 35 |
-
|
| 36 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
|
| 38 |
## Use
|
| 39 |
|
|
|
|
| 31 |
| int8 vs bf16 MLX | 48/48 tokens identical |
|
| 32 |
| final-position logits vs reference | cos 0.9982 (bf16 baseline: 0.9996) |
|
| 33 |
|
| 34 |
+
Measured on an M5 Max (macOS 27), one process per number, floor read post-load, at the default
|
| 35 |
+
4096-token budget:
|
| 36 |
+
|
| 37 |
+
| | resident floor | activation | decode |
|
| 38 |
+
|---|---|---|---|
|
| 39 |
+
| int8 · image | 5.0 GB | 3.0 GB | 85.7 tok/s |
|
| 40 |
+
| int8 · video 16f | 5.0 GB | 5.8 GB | 80.0 tok/s |
|
| 41 |
+
| bf16 · image | 8.85 GB | 2.85 GB | 51.2 tok/s |
|
| 42 |
+
| bf16 · video 16f | 8.85 GB | 5.8 GB | 50.2 tok/s |
|
| 43 |
+
|
| 44 |
+
Activation is unchanged by quantization (5.77 vs 5.79 GB) — only the resident floor moves.
|
| 45 |
+
|
| 46 |
+
The **token budget**, not the frame count, is the memory lever: at a fixed budget, 32 frames
|
| 47 |
+
costs slightly less than 16 because per-frame resolution drops to compensate. Raising the budget
|
| 48 |
+
is what costs — the same clip at an 8192-token budget needs **14.2 GB** of activation.
|
| 49 |
+
|
| 50 |
+
> **Corrected 2026-07-29.** This section previously read "~14 GB for 16-frame video QA at a
|
| 51 |
+
> 4096-token budget". That 14 GB figure is real but belongs to the *8192*-token budget: it was
|
| 52 |
+
> measured before per-frame video resolution was derived from the token budget, and was not
|
| 53 |
+
> re-measured when the default changed. Swift package `v0.3.0` re-declares its footprint
|
| 54 |
+
> accordingly.
|
| 55 |
|
| 56 |
## Use
|
| 57 |
|