π°π Khmer ASR SOTA β Complete 728h Open-Source Model
Khmer ASR SOTA is the first production-ready, open-source Automatic Speech Recognition (ASR) model trained on the complete 728-hour (653 shards) corpus of Cambodian Khmer speech.
Engineered by Vagabond404 (Angel Vision Labs) as the real-time, on-device transcription engine for Kinvoice (Ambient AI Second Brain & Voice Recorder).
π Key Achievements & Benchmarks
Compared to OpenAI Whisper Large-v2, Khmer ASR SOTA delivers superior accuracy on the Khmer language while operating at a fraction of the computational and memory footprint:
β’ Character Error Rate (CER) : 2.32% on test evaluation (dropping to 0.82% on clean speech domain), outperforming Whisper Large-v2 (~8.11% CER on paper) by over 3.5x.
β’ Word Error Rate (WER) : 53.44% overall (dropping to 29.49% on clean speech segments).
β’ Model Size & Edge AI : 338.6 MB (ONNX INT8 quantized) vs Whisper Large-v2 (1,550 MB) β 4.6x more compact, fitting easily into mobile app bundles (Flutter/Android/iOS).
β’ Real-Time Factor (RTF) : 0.05x - 0.16x on standard mobile/laptop CPUs (6x to 20x faster than real-time).
β’ Acoustic Stability : Non-autoregressive CTC architecture completely eliminates the repeating hallucinations, infinite loops, and silent drops common in Whisper on low-resource audio.
π¦ Training Corpus & Methodology
β’ Total Dataset Volume : 653 Parquet shards (~728 hours raw audio).
β’ Effective Filtered Audio : 437.51 hours of validated native Cambodian Khmer speech.
β’ Speech Samples Trained : > 180,000 distinct speech segments.
β’ Base Acoustic Architecture : Pretrained multilingual Wav2Vec2 / Meta MMS acoustic backbone.
β’ Vocabulary & Tokenizer : 75 pure Khmer tokens (covering all consonants, independent vowels, diacritics, subscript markers, and the word-boundary marker |).
β’ Hardware & Training Infrastructure : Dual Tesla T4 GPUs on Kaggle via PyTorch DDP (torchrun), FP16 mixed precision, dynamic padding collator, ctc_zero_infinity=True.
π Quickstart: Python Inference (Transformers)
import torch
import librosa
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
# 1. Load model and processor
model_id = "Vagabond404/khmer-asr-sota"
processor = Wav2Vec2Processor.from_pretrained(model_id)
model = Wav2Vec2ForCTC.from_pretrained(model_id)
model.eval()
# 2. Load and resample audio (must be 16,000 Hz mono)
audio_path = "sample_khmer.wav"
speech, sr = librosa.load(audio_path, sr=16000)
# 3. Predict transcription
inputs = processor(speech, sampling_rate=16000, return_tensors="pt", padding=True)
with torch.no_grad():
logits = model(inputs.input_values).logits
predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)[0]
print("π°π Transcription :", transcription)
β‘ Edge Inference: ONNX Runtime (Mobile & Desktop)
The repository provides khmer_asr_int8.onnx (quantized INT8), enabling lightning-fast CPU inference without PyTorch:
import numpy as np
import onnxruntime as ort
import librosa
from transformers import Wav2Vec2CTCTokenizer
# 1. Initialize ONNX session
session = ort.InferenceSession("khmer_asr_int8.onnx", providers=["CPUExecutionProvider"])
tokenizer = Wav2Vec2CTCTokenizer.from_pretrained("Vagabond404/khmer-asr-sota")
# 2. Load 16kHz audio
audio, _ = librosa.load("sample_khmer.wav", sr=16000)
input_values = np.expand_dims(audio.astype(np.float32), axis=0)
# 3. Run forward pass
inputs = {session.get_inputs()[0].name: input_values}
logits = session.run(None, inputs)[0]
# 4. Decode
pred_ids = np.argmax(logits, axis=-1)[0]
text = tokenizer.decode(pred_ids)
print("Transcription (ONNX) :", text)
π± Mobile & Kinvoice Integration (Flutter / Dart)
For Flutter mobile apps (Android & iOS via onnxruntime or tflite_flutter), feed raw PCM float32 buffers directly into the ONNX model inside a background Dart Isolate:
- Capture Microphone : 16 kHz Mono PCM float32 stream.
- Pre-filter VAD : Silero VAD filters non-speech frames.
- CTC Forward Pass : Runs in ~120 ms for a 5-second audio chunk.
- Argmax Decode : Map predicted indices through
vocab.json.
π Evaluation Evolution by Shard Blocks
The model underwent continuous checkpointing and evaluation across all 653 shards:
β’ Shards 0 β 100 (66h) : CER = 3.05% - 4.16% | WER = 59.78%120h) : CER = 1.73% | WER = 51.01%
β’ Shards 100 β 180 (
β’ Shards 180 β 260 (178h) : CER = 1.84% | WER = 49.56%232h) : CER = 1.30% | WER = 48.09%
β’ Shards 260 β 340 (
β’ Shards 340 β 420 (283h) : CER = 1.11% | WER = 44.63%337h) : CER = 1.60% | WER = 42.83%
β’ Shards 420 β 500 (
β’ Shards 500 β 580 (390h) : CER = 1.43% | WER = 42.41%437.5h) : CER = 0.82% (best) / 2.32% (global test) | WER = 29.49% (best) / 53.44% (global test)
β’ Shards 580 β 653 (FINAL) (
βοΈ License & Acknowledgments
β’ License : Apache 2.0 (Permissive for commercial and non-commercial research & products).
β’ Corpus Authors : Gratitude to Rina Buoy et al. (rinabuoy/khmer-speech-dataset) and Sobon Phon et al. (sobonphon/khmer-speech-dataset) for releasing open-access Khmer speech data.
β’ Developer : Vagabond404 / Angel Vision Labs (Phnom Penh / Siem Reap, Cambodia).
- Downloads last month
- 155
Datasets used to train Vagabond404/khmer-asr-sota
sobonphon/khmer-speech-dataset
Evaluation results
- Test CER (Character Error Rate) on Khmer Speech Dataset (653 Shards / 728 Hours)self-reported0.023
- Test WER (Word Error Rate) on Khmer Speech Dataset (653 Shards / 728 Hours)self-reported0.534