Instructions to use Pirnazar/mms-1b-turkmen with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Pirnazar/mms-1b-turkmen with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="Pirnazar/mms-1b-turkmen")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForCTC processor = AutoProcessor.from_pretrained("Pirnazar/mms-1b-turkmen") model = AutoModelForCTC.from_pretrained("Pirnazar/mms-1b-turkmen", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Turkmen MMS ASR
Turkmen speech โ Turkmen Latin text. A standalone Wav2Vec2/MMS CTC model
with one embedded tuk-script_latin adapter and one flat vocabulary.
It does not require downloading a separate language adapter or switching languages.
This is an experimental, non-commercial release by Pirnazar, based on Meta's facebook/mms-1b-all. It is a Transformers model, not a Whisper/CTranslate2 checkpoint.
Results
The exported checkpoint was selected using validation before the held-out test. Both the original Turkmen MMS adapter and this export were evaluated on the same audio and references.
| Evaluation | Original Turkmen MMS | This release |
|---|---|---|
| Held-out TurkmenSpeech test: strict WER | 37.05% | 24.58% |
| Held-out TurkmenSpeech test: strict CER | 5.95% | 4.11% |
- Held-out test: 3,879 clips, 8.42 hours, 61,733 reference words, 17 source recordings. None of those source recordings or full normalized transcript strings occur in the training or validation splits.
- Improvement was observed on all 17 test sources. Paired bootstrap over source recordings (2,000 resamples): 95% interval for the strict WER reduction is 11.72โ13.09 percentage points.
- Speaker identities are unknown. Source-disjoint evaluation does not establish speaker-disjoint evaluation or performance on every dialect.
- Strict metrics preserve case, punctuation and Turkmen diacritics after NFC normalization. Normalized metrics additionally casefold and map punctuation to spaces; diacritics are retained. CER includes spaces.
These figures are from this release's custom split, not an official benchmark leaderboard or a comparison against all Turkmen ASR systems. Conversational, social-media, noisy and musical audio remain challenging.
Local usage
Tested with Python 3.11/3.12, PyTorch 2.9.1+cu126, Transformers 5.17.0, and safetensors 0.8.0, on CPU and NVIDIA CUDA.
pip install "torch==2.9.1" "transformers==5.17.0" "safetensors==0.8.0" soundfile numpy
python transcribe.py speech.wav --device cpu
# With a suitable CUDA PyTorch build:
python transcribe.py speech.wav --device cuda
The demo expects 16 kHz WAV audio. Stereo input is averaged to mono. Convert other sample rates locally, for example:
ffmpeg -i recording.ogg -ar 16000 -ac 1 -c:a pcm_s16le speech.wav
For a model folder already downloaded to disk:
python transcribe.py speech.wav --model ./mms-1b-turkmen --local-files-only --device cpu
The first Hub load downloads weights. Recognition runs locally; no cloud STT or LLM correction is used. The demo processes long audio in overlapping windows.
Transformers example
from contextlib import nullcontext
import soundfile as sf
import torch
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
repo = "Pirnazar/mms-1b-turkmen"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.float16 if device == "cuda" else torch.float32
processor = Wav2Vec2Processor.from_pretrained(repo)
model = Wav2Vec2ForCTC.from_pretrained(
repo, dtype=dtype, attn_implementation="sdpa"
).to(device).eval()
audio, sr = sf.read("speech.wav", dtype="float32")
if audio.ndim == 2:
audio = audio.mean(axis=1)
if sr != 16000:
raise ValueError("Convert the audio to 16 kHz first")
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
inputs = {name: value.to(device) for name, value in inputs.items()}
amp = torch.autocast("cuda", dtype=dtype) if device == "cuda" else nullcontext()
with torch.inference_mode(), amp:
logits = model(**inputs).logits
text = processor.batch_decode(logits.argmax(-1))[0].strip()
print(text)
Use short utterances (up to approximately 30 seconds) with the direct example.
Use transcribe.py for longer recordings. Do not call load_adapter() or
set_target_lang() on this standalone release. No trust_remote_code is needed.
Training and provenance
- Base:
facebook/mms-1b-all, revision3d33597edbdaaba14a8e858e2c8caa76e3cec0cd. - The existing trained
tuk-script_latinadapter was preserved and further trained; it was not reinitialized. The backbone remained frozen. - Approximately 965M total parameters, 2.28M trainable adapter/head parameters.
- Training data: mamed0v/TurkmenSpeech.
- Train: 104,713 clips / 195.47 h / 370 sources. Validation: 6,146 clips / 11.36 h / 24 sources. Test: 3,879 clips / 8.42 h / 17 sources.
- Source grouping preceded duration filtering. Full clips above the duration bound were not shortened. 2,808 clips with repeated normalized transcript strings spanning splits were quarantined, including every cross-split copy.
- One epoch, 13,090 optimizer updates, microbatch 1, gradient accumulation 8, AdamW, learning rate 1e-4, weight decay 0.01, 5% warmup, linear decay with a 5%-of-peak floor, gradient clipping 1.0, seed 42, gradient checkpointing.
- The run continued from a laptop in BF16 to a second GPU in FP16, preserving adapter weights, AdamW state and data cursor. GradScaler was initialized for FP16. This was not a bitwise-identical cross-hardware continuation.
- Selected checkpoint: 11,000, using strict WER on a fixed 64-clip validation sample spanning all 24 validation sources. Validation WER: 24.84%.
- No private recordings were used for training. No LLM-generated corrections or synthetic additional training examples were added in this run.
- The vocabulary retains the original token IDs and appends characters for case, digits, punctuation and names. The output has 102 tokens. Character availability alone does not establish reliable brand-name recognition.
Public aggregate results and sanitized provenance are in evaluation_results.json
and training_provenance.json. No private audio, transcripts, user identifiers,
machine paths, optimizer state or credentials are included in this repository.
Limitations
- This is a single-language Turkmen Latin ASR model. Translation, diarization, speaker identification and reliable multilingual code switching are not provided.
- Dialects, informal endings, names and product brands can still be misrecognized.
- The training corpus and evaluation references may contain transcription errors.
- Runtime depends on hardware and audio length. The evaluation was on a working laptop and involved a process restart; its timings are not a controlled speed comparison. No universal latency or memory guarantee is made.
- This snapshot cannot be loaded by
faster-whisper; use Transformers/PyTorch.
License and attribution
CC BY-NC 4.0 โ non-commercial use only; attribution is required. See LICENSE and NOTICE.
Credit Meta / the MMS authors for the base weights, mamed0v for TurkmenSpeech, and Pirnazar for this adapter fine-tune and standalone export. This release changes the Turkmen adapter/output head and vocabulary and packages one language as a self-contained FP16 snapshot. The upstream model and dataset retain their respective notices and conditions.
- Downloads last month
- 45
Model tree for Pirnazar/mms-1b-turkmen
Base model
facebook/mms-1b-allEvaluation results
- WER (strict, percent) on TurkmenSpeech (custom source-disjoint held-out test)test set self-reported24.580
- CER (strict, percent) on TurkmenSpeech (custom source-disjoint held-out test)test set self-reported4.106