πŸ‡°πŸ‡­ Khmer ASR SOTA β€” Complete 728h Open-Source Model

Khmer ASR SOTA is the first production-ready, open-source Automatic Speech Recognition (ASR) model trained on the complete 728-hour (653 shards) corpus of Cambodian Khmer speech.

Engineered by Vagabond404 (Angel Vision Labs) as the real-time, on-device transcription engine for Kinvoice (Ambient AI Second Brain & Voice Recorder).


πŸ† Key Achievements & Benchmarks

Compared to OpenAI Whisper Large-v2, Khmer ASR SOTA delivers superior accuracy on the Khmer language while operating at a fraction of the computational and memory footprint:

β€’ Character Error Rate (CER) : 2.32% on test evaluation (dropping to 0.82% on clean speech domain), outperforming Whisper Large-v2 (~8.11% CER on paper) by over 3.5x.
β€’ Word Error Rate (WER) : 53.44% overall (dropping to 29.49% on clean speech segments).
β€’ Model Size & Edge AI : 338.6 MB (ONNX INT8 quantized) vs Whisper Large-v2 (1,550 MB) β€” 4.6x more compact, fitting easily into mobile app bundles (Flutter/Android/iOS).
β€’ Real-Time Factor (RTF) : 0.05x - 0.16x on standard mobile/laptop CPUs (6x to 20x faster than real-time).
β€’ Acoustic Stability : Non-autoregressive CTC architecture completely eliminates the repeating hallucinations, infinite loops, and silent drops common in Whisper on low-resource audio.


πŸ“¦ Training Corpus & Methodology

β€’ Total Dataset Volume : 653 Parquet shards (~728 hours raw audio).
β€’ Effective Filtered Audio : 437.51 hours of validated native Cambodian Khmer speech.
β€’ Speech Samples Trained : > 180,000 distinct speech segments.
β€’ Base Acoustic Architecture : Pretrained multilingual Wav2Vec2 / Meta MMS acoustic backbone.
β€’ Vocabulary & Tokenizer : 75 pure Khmer tokens (covering all consonants, independent vowels, diacritics, subscript markers, and the word-boundary marker |).
β€’ Hardware & Training Infrastructure : Dual Tesla T4 GPUs on Kaggle via PyTorch DDP (torchrun), FP16 mixed precision, dynamic padding collator, ctc_zero_infinity=True.


πŸš€ Quickstart: Python Inference (Transformers)

import torch
import librosa
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

# 1. Load model and processor
model_id = "Vagabond404/khmer-asr-sota"
processor = Wav2Vec2Processor.from_pretrained(model_id)
model = Wav2Vec2ForCTC.from_pretrained(model_id)
model.eval()

# 2. Load and resample audio (must be 16,000 Hz mono)
audio_path = "sample_khmer.wav"
speech, sr = librosa.load(audio_path, sr=16000)

# 3. Predict transcription
inputs = processor(speech, sampling_rate=16000, return_tensors="pt", padding=True)
with torch.no_grad():
    logits = model(inputs.input_values).logits

predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)[0]

print("πŸ‡°πŸ‡­ Transcription :", transcription)

⚑ Edge Inference: ONNX Runtime (Mobile & Desktop)

The repository provides khmer_asr_int8.onnx (quantized INT8), enabling lightning-fast CPU inference without PyTorch:

import numpy as np
import onnxruntime as ort
import librosa
from transformers import Wav2Vec2CTCTokenizer

# 1. Initialize ONNX session
session = ort.InferenceSession("khmer_asr_int8.onnx", providers=["CPUExecutionProvider"])
tokenizer = Wav2Vec2CTCTokenizer.from_pretrained("Vagabond404/khmer-asr-sota")

# 2. Load 16kHz audio
audio, _ = librosa.load("sample_khmer.wav", sr=16000)
input_values = np.expand_dims(audio.astype(np.float32), axis=0)

# 3. Run forward pass
inputs = {session.get_inputs()[0].name: input_values}
logits = session.run(None, inputs)[0]

# 4. Decode
pred_ids = np.argmax(logits, axis=-1)[0]
text = tokenizer.decode(pred_ids)
print("Transcription (ONNX) :", text)

πŸ“± Mobile & Kinvoice Integration (Flutter / Dart)

For Flutter mobile apps (Android & iOS via onnxruntime or tflite_flutter), feed raw PCM float32 buffers directly into the ONNX model inside a background Dart Isolate:

  1. Capture Microphone : 16 kHz Mono PCM float32 stream.
  2. Pre-filter VAD : Silero VAD filters non-speech frames.
  3. CTC Forward Pass : Runs in ~120 ms for a 5-second audio chunk.
  4. Argmax Decode : Map predicted indices through vocab.json.

πŸ“Š Evaluation Evolution by Shard Blocks

The model underwent continuous checkpointing and evaluation across all 653 shards:

β€’ Shards 0 βž” 100 (66h) : CER = 3.05% - 4.16% | WER = 59.78%
β€’ Shards 100 βž” 180 (
120h) : CER = 1.73% | WER = 51.01%
β€’ Shards 180 βž” 260 (178h) : CER = 1.84% | WER = 49.56%
β€’ Shards 260 βž” 340 (
232h) : CER = 1.30% | WER = 48.09%
β€’ Shards 340 βž” 420 (283h) : CER = 1.11% | WER = 44.63%
β€’ Shards 420 βž” 500 (
337h) : CER = 1.60% | WER = 42.83%
β€’ Shards 500 βž” 580 (390h) : CER = 1.43% | WER = 42.41%
β€’ Shards 580 βž” 653 (FINAL) (
437.5h) : CER = 0.82% (best) / 2.32% (global test) | WER = 29.49% (best) / 53.44% (global test)


βš–οΈ License & Acknowledgments

β€’ License : Apache 2.0 (Permissive for commercial and non-commercial research & products).
β€’ Corpus Authors : Gratitude to Rina Buoy et al. (rinabuoy/khmer-speech-dataset) and Sobon Phon et al. (sobonphon/khmer-speech-dataset) for releasing open-access Khmer speech data.
β€’ Developer : Vagabond404 / Angel Vision Labs (Phnom Penh / Siem Reap, Cambodia).

Downloads last month
155
Safetensors
Model size
0.3B params
Tensor type
F32
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Datasets used to train Vagabond404/khmer-asr-sota

Evaluation results

  • Test CER (Character Error Rate) on Khmer Speech Dataset (653 Shards / 728 Hours)
    self-reported
    0.023
  • Test WER (Word Error Rate) on Khmer Speech Dataset (653 Shards / 728 Hours)
    self-reported
    0.534