xocialize commited on
Commit
f1cebca
·
verified ·
1 Parent(s): cb39add

Correct the activation figure: 14 GB is the 8192-token budget, not the 4096 default

Browse files
Files changed (1) hide show
  1. README.md +21 -3
README.md CHANGED
@@ -31,9 +31,27 @@ Gated against the bf16 checkpoint and the PyTorch reference on the model's own
31
  | int8 vs bf16 MLX | 48/48 tokens identical |
32
  | final-position logits vs reference | cos 0.9982 (bf16 baseline: 0.9996) |
33
 
34
- Measured on an M5 Max (macOS 27): resident floor **5.0 GB** (bf16: 8.85), image-QA decode
35
- **88.3 tok/s** (bf16: 54.9). Activation memory is unchanged by quantization — budget
36
- ~3 GB for a single image and ~14 GB for 16-frame video QA at a 4096-token budget.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
 
38
  ## Use
39
 
 
31
  | int8 vs bf16 MLX | 48/48 tokens identical |
32
  | final-position logits vs reference | cos 0.9982 (bf16 baseline: 0.9996) |
33
 
34
+ Measured on an M5 Max (macOS 27), one process per number, floor read post-load, at the default
35
+ 4096-token budget:
36
+
37
+ | | resident floor | activation | decode |
38
+ |---|---|---|---|
39
+ | int8 · image | 5.0 GB | 3.0 GB | 85.7 tok/s |
40
+ | int8 · video 16f | 5.0 GB | 5.8 GB | 80.0 tok/s |
41
+ | bf16 · image | 8.85 GB | 2.85 GB | 51.2 tok/s |
42
+ | bf16 · video 16f | 8.85 GB | 5.8 GB | 50.2 tok/s |
43
+
44
+ Activation is unchanged by quantization (5.77 vs 5.79 GB) — only the resident floor moves.
45
+
46
+ The **token budget**, not the frame count, is the memory lever: at a fixed budget, 32 frames
47
+ costs slightly less than 16 because per-frame resolution drops to compensate. Raising the budget
48
+ is what costs — the same clip at an 8192-token budget needs **14.2 GB** of activation.
49
+
50
+ > **Corrected 2026-07-29.** This section previously read "~14 GB for 16-frame video QA at a
51
+ > 4096-token budget". That 14 GB figure is real but belongs to the *8192*-token budget: it was
52
+ > measured before per-frame video resolution was derived from the token budget, and was not
53
+ > re-measured when the default changed. Swift package `v0.3.0` re-declares its footprint
54
+ > accordingly.
55
 
56
  ## Use
57