al-bo/gigaam-v3-rnnt-mlx

GigaAM v3 RNNT — MLX (Apple Silicon)

MLX conversion of GigaAM v3 RNNT for on-device Russian speech recognition on Apple Silicon. Character-level RNNT model (33 Russian letters + blank): lowercase text without punctuation. Lower WER than CTC, sequential greedy decode.

• Source: ai-sage/GigaAM-v3, rnnt revision
• Architecture: 16-layer Conformer encoder (d_model 768, 16 heads, rotary positions)
• Input: mono 16 kHz audio

Usage

The runtime and converter live in mlx_convert/ of misteral/GigaAM (proposed upstream in salute-developers/GigaAM#62).

git clone -b feat/mlx-clean https://github.com/misteral/GigaAM && cd GigaAM/mlx_convert
pip install mlx safetensors numpy huggingface_hub   # ffmpeg is used by load_audio
hf download al-bo/gigaam-v3-rnnt-mlx --local-dir ./gigaam-v3-rnnt-mlx

from gigaam_mlx import load_model, load_audio   # mlx_convert/gigaam_mlx.py

model = load_model("./gigaam-v3-rnnt-mlx")
print(model.transcribe(load_audio("audio.wav")))

Evaluation

Apple M4 Pro (24 GB), mlx 0.32.2, torch 2.14.0; 200 Golos Crowd test clips (823.85 s) — the sample of garrrikkotua/gigaam-vs-whisper-russian, verified by SHA-256. Method and full tables: mlx_convert/eval.

• Parity with upstream PyTorch (CPU, fp32): identical transcripts on 200/200 clips; encoder output max |Δ| 3.2e-06 median / 1.7e-05 max per clip.
• WER 1.50% (15 errors / 1001 words), CER 0.24%.
• Speed, × realtime (200 clips one by one, median 3.9 s / ≈20 s segments): MLX 91 / 102, PyTorch MPS 37 / 40, PyTorch CPU 10 / 39.

Files

• model.safetensors — fp16 MLX weights (423 MB)
• config.json — architecture + vocabulary

The upstream checkpoint is stored in fp16, so the fp16 weights here lose nothing on the encoder.

Notes for other runtimes

If you run these weights with your own implementation, two details differ from a textbook Conformer:

• RoPE is applied to the raw input, split into heads, before the Q/K projections (value is not rotated).
• The RoPE base is 5000 (pos_emb_max_len), not 10000: upstream calls RotaryPositionalEmbedding(d_model // n_heads, pos_emb_max_len) and the second positional parameter is base. With 10000 transcripts still match on most clips, but encoder outputs are off by up to ~1.8.

Other details: log-mel uses mel_filterbank and stft_window stored in model.safetensors (taken from the PyTorch model, center=False, log(clip(x, 1e-9, 1e9))); Conv1d weights are in MLX layout [out, K, in]; the LSTM bias is bias_ih + bias_hh; blank = len(vocabulary).

License

Model weights: MIT, per the upstream ai-sage/GigaAM-v3 card. Conversion code: MIT. Original project: salute-developers/GigaAM.