kaan84/sinhala-print-trocr

TrOCR Sinhala Printed Lines

Phase 1 in progress. Two of three pre-registered confirmatory runs are complete. Exploratory pilot weights are released. No real-document evaluation results exist yet.

This project investigates whether fine-tuning Ransaka/TrOCR-Sinhala on synthetic Sinhala text-line images improves printed, line-level Sinhala OCR performance on real documents.

Project status

Workstream	Status
Research scope	Locked in PROTOCOL.md v1.1
Dataset license	Confirmed MIT
Intermediate checkpoint license	Confirmed MIT (author, 2026-07-30)
Pilot run	Complete — exploratory, reported separately
Confirmatory training	Seeds 42 and 123 complete; seed 2026 pending
Real-document evaluation	Not started
Model weights	Pilot published; confirmatory weights pending

The complete model and training-data chain is MIT-licensed: microsoft/trocr-base-printed → Ransaka/TrOCR-Sinhala → Ransaka/sinhala_synthetic_ocr-large → this project's derived weights. Ransaka confirmed the intermediate checkpoint's MIT license and declared it in the repository metadata. The permanent confirmation record is linked in ATTRIBUTION_AND_LICENSING.md.

Exploratory pilot weights: kaan84/sinhala-print-trocr-pilot. Pilot results are never pooled with confirmatory results; see PROTOCOL.md Section 2.

Research question

Does synthetic-data fine-tuning of TrOCR improve printed, line-level Sinhala OCR on real documents?

Scope

In scope

• Printed Sinhala text
• Line-level text recognition
• Character Error Rate (CER)
• Word Error Rate (WER)
• Comparisons with the unmodified checkpoint and Tesseract 5.5
• Optional Surya comparison when its applicable terms are confirmed
• Period-stratified analysis where document-year metadata is available

Out of scope

• Handwriting
• Full-page layout detection or reading order
• Production or high-stakes deployment
• Broad Sinhala-English code mixing
• Claims of parity with page-level commercial or large vision-language systems

Data

Training and validation

Ransaka/sinhala_synthetic_ocr-large

• 6,969 synthetic Sinhala image-text pairs
• One published train split
• Five reported font families
• Dataset DOI: 10.57967/hf/1809
• Repository LICENSE: MIT
• Fixed project split: 90% train and 10% validation, permuted under the run seed
• Rows with null or empty reference text are removed before splitting (one row removed: 6,969 → 6,968)

Per-run manifests in results/ record the immutable asset revisions, the split-index file, its SHA-256 checksum, and the full dependency versions.

Real-document evaluation

avishadilhara/sinhala-ocr-lk-acts-1010

• 1,010 page images from Sri Lankan legislative acts
• Published splits: train 707, test 202, evaluation 101
• License: CC BY 4.0
• Paper: arXiv:2606.29378

This dataset is listed in repository metadata for discoverability but is used for evaluation only and never for training.

TrOCR is a line-recognition model. The page dataset must therefore be converted into a documented, deterministic line-level benchmark before direct evaluation. Page-level benchmark figures are contextual references and must not be presented as directly comparable line-level results.

Pre-registered experiment

The hypotheses, fixed splits, seeds, metrics, normalization, hyperparameters, decision rules, and reporting requirements are defined in PROTOCOL.md. Material deviations must be documented before results are interpreted; the deviation log is Section 17.

Planned baselines

System	Role	Primary comparison
Ransaka/TrOCR-Sinhala	Starting checkpoint	Measures improvement from additional fine-tuning
Tesseract 5.5	Permissive classical baseline	CER and WER on the same line images
Surya OCR	Optional research baseline	CER and WER on the same evaluation unit

Published page-level results are contextual targets only:

System	CER	WER	Evaluation unit
Tesseract v5	10.69%	Not reported	Real pages
Surya OCR	8.84%	26.64%	Real pages
Google Document AI	2.06%	Not reported	Real pages
Fine-tuned LightOnOCR-2-1B	1.05%	5.63%	Real pages

These figures come from a June 2026 preprint and are not direct line-level acceptance thresholds.

Pre-registered training configuration

Parameter	Pre-registered value
Optimizer	AdamW
Learning rate	4e-5
Epochs	10
Effective batch size	8
Precision	FP16 when supported
Seeds	42, 123, 2026
Evaluation beams	4
Runs	3 independent runs

The machine-readable configuration is in configs/phase0.yaml.

Results

Real-document benchmark — not yet run

System	Dataset/slice	CER mean ± SD	WER mean ± SD	Status
Starting checkpoint	Real Sinhala line benchmark	TBD	TBD	Not run
Fine-tuned model	Real Sinhala line benchmark	TBD	TBD	Not run
Tesseract 5.5	Real Sinhala line benchmark	TBD	TBD	Not run
Surya OCR	Real Sinhala line benchmark	TBD	TBD	Optional

This table answers the research question. It is empty because the line-level benchmark has not been constructed. No performance claim is made until it is populated.

Synthetic validation — not a performance claim

Held-out synthetic validation, 697 samples, same generator and fonts as the training data. These figures measure fit to one synthetic renderer, not Sinhala reading ability.

Run	Type	Seed	CER	WER	Exact match
Confirmatory	Pre-registered	42	14.39%	28.18%	6.89%
Confirmatory	Pre-registered	123	14.75%	28.92%	4.88%
Confirmatory	Pre-registered	2026	Pending	Pending	Pending
Mean ± SD			Pending all three	Pending all three	Pending all three
Pilot	Exploratory	42	16.98%	32.11%	3.87%

The pilot used a different learning rate, epoch count, and split, and its figures come from a synthetic test split rather than a validation split. Pilot and confirmatory rows are not comparable and must not be pooled.

Observations after two confirmatory runs, to be confirmed by the third:

• Run-to-run stability. CER spread across seeds is 0.36 percentage points, within the 1-point instability threshold in PROTOCOL.md Section 17. Exact match varies more (6.89% against 4.88%) because a single joiner or spacing difference fails an entire line.
• No convergence plateau. Validation CER improved through the final epoch in every run so far, indicating the model is training-budget-limited rather than converged at this data scale. The epoch count remains fixed at the pre-registered value; this observation motivates later data-scale work rather than a change to the current experiment.
• Raw and normalized metrics are identical on synthetic validation, because tokenizer decoding already yields NFC-normalized, whitespace-collapsed text. The two conventions are expected to diverge only in the real-benchmark comparison described in PROTOCOL.md Section 12.1.

Do not add zero-value placeholders to Hugging Face model-index metadata; they can be mistaken for real results.

Evaluation and reporting

CER is the primary metric and WER is secondary. Report:

• Results for every seed, plus mean and standard deviation
• Exact dataset revisions and split-index checksum
• Raw and normalized metrics
• 95% paired bootstrap confidence intervals for model differences
• Error examples grouped by degradation type
• Results by document period where supported
• Runtime and hardware details

Synthetic validation metrics decode predictions and references through the model tokenizer, so its spacing and ZWJ/ZWNJ losses cancel on both sides. Real-benchmark metrics are computed against raw ground-truth text, because external baselines do not pass through this tokenizer. See PROTOCOL.md Section 12.1.

Intended use

The eventual model is intended for research and prototyping involving printed Sinhala text lines, including OCR experiments, document digitization research, and accessibility prototypes. Human review is required for consequential use.

Limitations

• Synthetic training data may not represent real scanning noise, skew, bleed-through, compression, damaged print, or historical typography.
• Training fonts cover only a limited part of Sinhala typography.
• The inherited tokenizer does not preserve Zero Width Joiner and Zero Width Non-Joiner characters, which affects roughly 6.6% of training texts and places a ceiling on Sinhala conjunct output. See PROTOCOL.md Section 2.2.
• Exact-match rates below 7% on clean synthetic data indicate most predicted lines still contain at least one character error.
• Accuracy may degrade on older documents, handwriting, mixed-language text, unusual symbols, curved lines, and full pages.
• Line segmentation quality can dominate end-to-end page performance.
• Results from legislative acts may not generalize to receipts, books, forms, signs, or contemporary digital screenshots.

Bias and responsible use

The training domain may overrepresent clean, computer-generated Sinhala and underrepresent historical fonts and real capture conditions. Evaluate new domains before use. Do not use unreviewed OCR output as authoritative evidence in legal, medical, financial, identity, or public-administration decisions.

Public legislative documents can contain names. Preserve source attribution, minimize unnecessary retention, and review extracted text before redistribution.

Inference

Confirmatory weights are not published in this repository yet. The pilot weights load as follows:

from PIL import Image
from transformers import TrOCRProcessor, VisionEncoderDecoderModel

repo_id = "kaan84/sinhala-print-trocr-pilot"
processor = TrOCRProcessor.from_pretrained(repo_id)
model = VisionEncoderDecoderModel.from_pretrained(repo_id)

image = Image.open("sinhala-line.png").convert("RGB")
pixel_values = processor(images=image, return_tensors="pt").pixel_values
generated_ids = model.generate(pixel_values, num_beams=4)
text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(text)

Input must be a cropped single line of printed Sinhala text. Auto-generated AutoModel or pipeline("image-to-text") snippets do not load this architecture correctly.

Repository structure

.
├── README.md
├── PROTOCOL.md
├── RISK_REGISTER.md
├── ATTRIBUTION_AND_LICENSING.md
├── REPRODUCIBILITY_CHECKLIST.md
├── requirements.txt
├── configs/
│   └── phase0.yaml
├── scripts/
│   └── README.md
├── assets/
│   └── README.md
├── runs/
│   └── README.md
└── results/
    ├── README.md
    ├── metrics.template.json
    ├── manifest-seed-123.json
    └── split-indices-seed-123.json

Release gates

1. ✅ Obtain and archive explicit licensing clarification for Ransaka/TrOCR-Sinhala.
2. ✅ Record immutable dataset and model revisions (per-run manifests).
3. ✅ Commit split indices and their checksum.
4. ⬜ Complete three seeded runs (2 of 3 done).
5. ⬜ Run every baseline on identical evaluation units.
6. ⬜ Populate real metrics and add model-index metadata.
7. ⬜ Complete the reproducibility checklist.
8. ✅ Publish model weights only after the licensing gate passes (pilot weights released under the confirmed MIT chain).

Gates 2 and 3 were satisfied from the seed 123 run onward. The seed 42 confirmatory run predates them; see PROTOCOL.md deviations D-2, D-3, and D-4.

Citation

If this research is published, add its final citation here. Upstream citations and licensing notes are maintained in ATTRIBUTION_AND_LICENSING.md.