kaan84/whisper-small-sinhala-v1

Whisper Small Sinhala v1

whisper-small-sinhala-v1 is a fine-tuned version of OpenAI Whisper Small for automatic speech recognition (ASR) in Sinhala. It is intended to convert Sinhala speech into written Sinhala text.

This repository is currently gated. Request access through the model page before downloading the weights.

Model details

Item	Value
Task	Automatic speech recognition
Language	Sinhala (si)
Base model	openai/whisper-small
Architecture	Whisper
Parameters	Approximately 241.7M
Weight format	Safetensors

Intended use

The model is suitable for research, prototyping, and Sinhala speech-transcription experiments. Potential applications include transcription tools, subtitle preparation, voice interfaces, and accessibility workflows.

Performance can vary with accent, background noise, recording quality, speaking rate, and domain-specific vocabulary. Review generated transcripts before using them in high-impact workflows.

Usage

import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline

model_id = "kaan84/whisper-small-sinhala-v1"
device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id,
    torch_dtype=torch_dtype,
    low_cpu_mem_usage=True,
    use_safetensors=True,
)
model.to(device)

processor = AutoProcessor.from_pretrained(model_id)

asr = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    torch_dtype=torch_dtype,
    device=device,
)

result = asr("sample.wav", generate_kwargs={"language": "si", "task": "transcribe"})
print(result["text"])

Training data

The repository metadata does not currently provide a complete dataset card. Add the following before treating this model as production-ready documentation:

• Dataset name, source, and version
• Number of audio hours and speakers
• Train, validation, and test split sizes
• Audio preprocessing and filtering steps
• Licensing and consent information

Training procedure

This model was fine-tuned from openai/whisper-small. Training hyperparameters, hardware, checkpoint-selection criteria, and preprocessing settings should be added here to make the result reproducible.

Evaluation

The model was evaluated during training on a fixed test subset of the Sinhala ASR dataset.

Evaluation setup

Item	Value
Training samples	50,000
Evaluation samples	10,000
Training steps	2,000
Effective epochs	0.64
Batch size	16
Learning rate	1e-5
Evaluation interval	Every 100 steps

Results

Metric	Result
Final validation loss	0.1546
Final step training loss	0.1515
Overall training loss	0.2706
Training runtime	2,220.99 seconds
Training samples/second	14.41

Validation loss decreased from 0.7160 at step 100 to 0.1546 at step 2,000, indicating consistent improvement throughout training.

Qualitative example

Reference

මේ නිසා බොහෝ දෙනකු

Prediction

මේ නිසා බොහෝ දෙනෙකු

For this individual example, the word error rate was 25.0% and the character error rate was approximately 5.6%. The difference is a minor spelling variation.

Evaluation limitations

Corpus-level WER and CER were not calculated across the complete 10,000-sample evaluation subset. Therefore, the example-level error rates above should not be interpreted as the model’s overall performance.

The model remains research-grade. Performance may vary depending on speaker accent, recording quality, background noise, audio duration, and domain-specific vocabulary.

Limitations

• The model may produce incorrect or incomplete transcriptions.
• Accuracy may decrease for noisy, overlapping, distant, or low-quality speech.
• Coverage may be uneven across Sinhala accents, regions, ages, and specialized domains.
• The model may reproduce biases or errors present in its training data.
• Human verification is recommended for legal, medical, educational-assessment, or other consequential use.

License

Apache License 2.0. Confirm that all training datasets and redistributed assets are compatible with this license.

Citation

If you use this model, cite the original Whisper paper and link to this model repository. Add a project-specific BibTeX entry when the training work is formally published.