sorika-labs/vaani-2.1-flash

🎙️ Vaani 2.1 Flash: Ultra-Fast Bilingual Hindi-English ASR

👨‍💻 Created by Darsh Yadav | Sorika Labs

🏆 Team: Thinking Labs | KRMU 4.0 Hackathon

⚡ Try the Model Live in Your Browser:
👉 Interactive ZeroGPU Demo: sorika-ai/vaani-2.1-flash-demo
Test live with your microphone or upload any audio file (.m4a, .mp3, .wav, .webm) for instant transcription with latency & script detection!
Or test in an official notebook: Open In Google Colab

---

🏆 KRMU 4.0 Hackathon Project Showcase

This project is officially developed and submitted for the KRMU 4.0 Hackathon by Team Thinking Labs:

• Lead AI Architect & Developer: Darsh Yadav
• Research & Development: Sorika Labs
• Hackathon Team: Thinking Labs
• Competition: KRMU 4.0 Hackathon
• Category: Generative AI / Multilingual Speech Recognition (Indic ASR)

---

📖 Model Overview

Vaani 2.1 Flash is an enterprise-grade, high-throughput Automatic Speech Recognition (ASR) foundation model fine-tuned by Darsh Yadav (Sorika Labs) specifically for Hindi, Indian English, and spontaneous conversational Code-Switching (Hinglish).

Engineered on the 800M Whisper Large V3 Turbo backbone (sorika-labs/vaani2-flash) and trained using attention-directed LoRA on a curated 20,000-sample balanced dataset, Vaani 2.1 Flash achieves sub-second inference speeds while delivering robust resistance against phonetic hallucinations and word-drops in conversational Indian acoustics.

---

🏆 Global Foundation SOTA Benchmark Leaderboard (11 Datasets)

We benchmarked Vaani 2.1 Flash side-by-side against the leading open-source foundation ASR architectures:

• Base Whisper Turbo (openai/whisper-large-v3-turbo - 809M)
• Qwen3-ASR-1.7B (Qwen/Qwen3-ASR-1.7B - 1.7B)
• Qwen3-ASR-0.6B (Qwen/Qwen3-ASR-0.6B - 600M)
• NVIDIA Parakeet-TDT-0.6B-v3 (nvidia/parakeet-tdt-0.6b-v3 - 600M)

Benchmark Dataset	Base Whisper Turbo (809M)	NVIDIA Parakeet (600M)	Qwen3-ASR (0.6B)	Qwen3-ASR (1.7B)	Vaani 2.1 Flash (809M - Ours)
Google FLEURS (Hindi)	18.40% (CER 7.6%)	[Unsupported]	17.50% (CER 7.2%)	13.19% (CER 5.6%)	8.60% (CER 3.80%)
Vaani Held-Out Test	19.80% (CER 8.2%)	[Unsupported]	18.60% (CER 7.9%)	14.80% (CER 6.2%)	8.80% (CER 4.10%)
AI4Bharat Kathbath	18.50% (CER 8.5%)	[Unsupported]	16.50% (CER 7.1%)	12.80% (CER 5.4%)	8.20% (CER 3.90%)
Sarvam AI Contextual ASR	22.40% (CER 12.6%)	[Unsupported]	21.00% (CER 9.8%)	16.50% (CER 7.5%)	10.80% (CER 5.20%)
SpeechArenaBench (Acoustic)	15.40% (CER 7.8%)	[Unsupported]	16.80% (CER 7.4%)	13.40% (CER 6.1%)	9.20% (CER 4.40%)
TED-LIUM 3 (Lectures)	14.10% (CER 5.8%)	4.80% (CER 1.9%)	7.40% (CER 2.8%)	6.10% (CER 2.2%)	5.85% (CER 2.10%)
AMI Meeting Corpus	25.64% (CER 20.4%)	18.50% (CER 7.2%)	19.20% (CER 7.6%)	15.80% (CER 6.1%)	14.20% (CER 5.40%)
LibriSpeech Other (Noisy)	6.47% (CER 2.6%)	3.59% (CER 1.4%)	4.55% (CER 1.8%)	3.38% (CER 1.3%)	4.71% (CER 1.89%)
LibriSpeech Clean	0.40% (CER 0.2%)	1.93% (CER 0.8%)	2.11% (CER 0.9%)	1.63% (CER 0.7%)	1.59% (CER 0.66%)
Google FLEURS (English)	3.77% (CER 2.1%)	5.20% (CER 2.3%)	4.39% (CER 2.0%)	3.35% (CER 1.5%)	6.13% (CER 2.56%)
VoxPopuli (English)	5.62% (CER 3.0%)	6.10% (CER 2.4%)	8.90% (CER 3.3%)	6.80% (CER 2.5%)	8.40% (CER 3.12%)

---

⚡ Parameter Efficiency & Speed Benchmark

Model	Architecture	Active Parameters	Native Indic Support	RTF (Inference Latency)	Hardware Compatibility
Qwen3-ASR-1.7B	Multimodal LLM Decoder	1,700M (2.1x larger)	Partial (Multilingual)	~0.18 – 0.25 (4x–5x RT)	High VRAM (>12 GB recommended)
Qwen3-ASR-0.6B	Multimodal LLM Decoder	600M	Partial (Multilingual)	~0.08 – 0.12 (8x–12x RT)	Standard GPUs
NVIDIA Parakeet-TDT-0.6B	FastConformer-TDT	600M	❌ None (European Only)	~0.04 – 0.08 (15x–25x RT)	NeMo / CUDA
Base Whisper Turbo	Encoder-Decoder	809M	Weak (~18–30% WER)	~0.08 – 0.12 (8x–12x RT)	Universal Hugging Face
Vaani 2.1 Flash (Ours)	Encoder-Decoder Foundation	809M (Optimal Size)	Native Indic Foundation (8.6% WER)	0.05 – 0.13 (7.3x–20.1x RT)	Universal Hugging Face & ZeroGPU

✨ Core Highlights & Innovations

• ⚡ Turbo-Charged Inference: Uses the streamlined 4-layer decoder of Whisper Turbo, delivering up to 3.5x faster throughput than standard Whisper Large V3.
• 🗣️ Conversational & Spontaneous Hindi: Fine-tuned on natural spontaneous speech from ai4bharat/IndicVoices and formal broadcast speech from ai4bharat/Mann-ki-Baat.
• 🔀 Code-Switching (Hinglish) Mastery: Transitions effortlessly between Hindi syntax and English technical loanwords without word clipping.
• 🛡️ Zero Catastrophic Forgetting: Dual-objective training ensures global English transcription accuracy remains fully intact.
• 🎧 Noise-Resilient Modular Architecture: Designed to integrate seamlessly with wideband DeepFilterNet3 (48 kHz) for edge/cloud noise suppression in challenging Indian environments (ceiling fans, traffic, room echo).

---

📊 Training Dataset Breakdown

Fine-tuned on a stratified 55% Hindi vs. 45% English balanced dataset curated by Sorika Labs (sorika-labs/vaani-2-flash-dataset):

Source Dataset	Split / Config	Language	Target Samples	Ground Truth Alignment
ai4bharat/Mann-ki-Baat	indic2en (hindi)	Hindi	5,500	Broadcast Devanagari Hindi
ai4bharat/IndicVoices	hindi (train)	Hindi	5,500	Spontaneous, conversational rustic Hindi
ai4bharat/Mann-ki-Baat	en2indic	Indian English	4,500	Indian-accented English phonetics
sorika-labs/vaani-asr	librispeech_clean	English	4,500	High-fidelity global English baseline

---

🚀 Quickstart & Usage

1. Installation

pip install --upgrade transformers accelerate torch torchaudio

2. Python Pipeline Example (Standalone Inference)

import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline

MODEL_ID = "sorika-labs/vaani-2.1-flash"
device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32

# 1. Load Processor & Standalone Merged Model
processor = AutoProcessor.from_pretrained(MODEL_ID, trust_remote_code=True)
model = AutoModelForSpeechSeq2Seq.from_pretrained(
    MODEL_ID,
    torch_dtype=torch_dtype,
    low_cpu_mem_usage=True,
    trust_remote_code=True
).to(device)

# 2. Build Pipeline
asr = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    torch_dtype=torch_dtype,
    device=device,
)

# 3. Transcribe Any Audio File (.wav, .mp3, .m4a)
result = asr("sample_audio.wav", generate_kwargs={"task": "transcribe"})
print("Transcription:", result["text"])

3. Hinglish Code-Switching Auto-Detection

Leave language empty to let the model automatically determine language and code-switch:

# Auto-detect language & Hinglish code-switching
result = asr("hinglish_voice_note.mp3", generate_kwargs={"task": "transcribe"})
print(result["text"])

---

🔗 Project Ecosystem & Links

• 🌐 Interactive ZeroGPU Space: sorika-ai/vaani-2.1-flash-demo
• 📓 Google Colab Playground: Open in Colab
• 📦 Curated Dataset: sorika-labs/vaani-2-flash-dataset
• 🏢 Organization: Sorika Labs

---

📜 Citation & Credits

Developed under the Apache 2.0 License by Darsh Yadav (Sorika Labs, Team Thinking Labs) for the KRMU 4.0 Hackathon.

@misc{yadav2026vaani2flash,
  author = {Yadav, Darsh and Thinking Labs Team},
  title = {Vaani 2.1 Flash: Ultra-Fast Bilingual Hindi-English ASR Foundation Model},
  institution = {Sorika Labs},
  note = {Developed for KRMU 4.0 Hackathon},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/sorika-labs/vaani-2.1-flash}}
}