jais-6.7b · FP8 · calibrated on 64 MSA + 64 Gulf

Post-training W8A8 quantization with NVIDIA ModelOpt (FP8_DEFAULT_CFG).

base model inceptionai/jais-family-6p7b-chat
base dtype bfloat16
precision FP8 (W8A8)
calibration variety 64 MSA + 64 Gulf
calibration file calib3_mixed.txt
calibration samples 128 dialogues, truncated at 128 tokens
calibration source Almheiri/ArabCulture-Dialogue rev 9acd60cbbb4f, seed 1448
weight MSE 7.106e-07
activation quantizers calibrated 225/225
environment torch 2.8.0+cu128 · transformers 4.57.6 · modelopt 0.45.0

FP8 weight scales are data-free. What the calibration variety changes is the static per-tensor activation scales, exported as input_scale. Sibling checkpoints -fp8-msa, -fp8-gulf and -fp8-mixed differ in exactly that and nothing else.

Not loadable by plain transformers — config.json declares quantization type modelopt.

vllm serve NouraAlqasim/jais-6.7b-fp8-mixed --quantization modelopt
Downloads last month
15
Safetensors
Model size
7B params
Tensor type
BF16
·
F8_E4M3
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for NouraAlqasim/jais-6.7b-fp8-mixed

Quantized
(5)
this model

Collection including NouraAlqasim/jais-6.7b-fp8-mixed