espnet/geolid_vl107only_independent_trainable

ESPnet2 GeoLID: vl107only_independent_trainable

This is a normal ESPnet language identification model using the stock S3prlFrontend and ESPnetLIDModel (model: espnet) with optional lang2vec conditioning. GeoLID wraps SSL layers 32, 36, 40 and 44 with lang2vec prediction and residual conditioning. The model code is maintained in ESPnet; no checkpoint-local Python modeling files or trust_remote_code=True are needed.

The checkpoint was converted, not retrained. local/convert_checkpoint.py renamed the original parameter keys and removed checked duplicate aliases. All learned tensor values are unchanged. The auxiliary linear head now belongs to the model; classification uses the standard aamsoftmax_sc_topk loss. Runtime uses current ESPnet pooling and normalization, without legacy compatibility code. Measured accuracy of this converted checkpoint is reported separately from historical paper results below.

Evaluation

Dataset	Historical paper accuracy (%)	Converted-model accuracy (%)	Correct / total
VoxLingua107 dev	93.7	93.78	1509 / 1609

The converted checkpoint was evaluated again on all 1,609 VoxLingua107 dev utterances after integrating lang2vec into the standard ESPnetLIDModel. This full evaluation used the unmodified espnet2.bin.lid_inference CLI at ESPnet revision 301372e4da311d2a3d9574a5a43b3037ae6e67c1: GPU float32, batch size 1, 8 data-loading workers, full-length waveforms, --use_preprocessor false and --fix_duration false. The input wav.scp was sorted by utterance ID. The environment used PyTorch 2.11.0+cu128, Transformers 5.14.1 and espnet-s3prl 2026.5.14. Accuracy is the fraction of correctly classified utterances.

Evaluation report and reproduction command, all 1,609 predictions with reference labels, inference log, and evaluation metadata provide the results, runtime settings, input revision and checkpoint SHA-256. All 1,609 predicted labels match the earlier full-dev evaluation at ca777d59eb49519e3827e40cc34997f2123d6682. The previous evaluation metadata is preserved in evaluation_previous.json.

These are batch-size-1 results. utterance_mvn and final chn_attn_stat pooling do mask padding. The remaining batch dependence comes earlier: S3PRL aligns features to ceil(samples / 320), so the extra frames of a shorter utterance can differ from its single-utterance run, and ECAPA's temporal convolutions and unmasked SE global means depend on padded sequence length. Final masked pooling cannot undo those feature changes. Historical paper scores for other datasets are not new measurements of this checkpoint.

The configuration selects the standard ESPnetLIDModel; lang2vec prediction and conditioning are optional features of that model. The refactor preserved the converted parameter names and weight file. The earlier strict-loading and six-utterance checks remain recorded at their original revisions in validation.json. Its separate full_dev_model_integration record describes this new complete evaluation with the stock CLI loader; the full evaluation did not instrument that loader to force strict loading.

Installation

Until PR #6212 is merged, install the implementation revision:

pip install 'espnet @ git+https://github.com/Qingzheng-Wang/espnet.git@301372e4da311d2a3d9574a5a43b3037ae6e67c1'
pip install espnet-s3prl transformers espnet-model-zoo soundfile

S3PRL's hf_wav2vec2_custom upstream imports Transformers; espnet-s3prl does not install it automatically. With an ESPnet tools checkout, the equivalent installers are make s3prl.done transformers.done in tools/. The lang2vec package is used only by recipe target preparation and is not needed for inference.

Inference

import soundfile as sf
import torch
from espnet_model_zoo.downloader import ModelDownloader
from espnet2.tasks.lid import LIDTask

files = ModelDownloader().download_and_unpack("espnet/geolid_vl107only_independent_trainable")
model, args = LIDTask.build_model_from_file(
    files["train_config"], files["model_file"], device="cpu"
)
model.eval()
audio, sample_rate = sf.read("speech.wav", dtype="float32")
assert sample_rate == 16000 and audio.ndim == 1
with torch.inference_mode():
    embeddings, prediction = model(
        torch.from_numpy(audio)[None],
        torch.tensor([len(audio)]),
        extract_embd=True,
    )
with open(args.lang2utt) as stream:
    languages = [line.split()[0] for line in stream]
print(languages[prediction.item()])

The backbone is initialized through the stock hf_wav2vec2_custom upstream from facebook/mms-1b, then the complete converted checkpoint loads the trained SSL, conditioning and classifier parameters. This repository contains 107 language classes. languages.txt preserves classifier-index order recovered from the original model card and the original sorted recipe label construction; it is not a copy of the missing original training utterance manifest.

conversion.json records the source checkpoint revision and label provenance. validation.json preserves the historical strict-loading and layout-migration checks, and separately records the complete evaluation after model integration. evaluation.json records the latest measured accuracy, with per-utterance results in evaluation_predictions.jsonl. The metadata targets are used only for training; inference uses speech and predicts its own conditioning vectors. The old training logs and figures retained under exp_* describe the original training run. Earlier checkpoint layouts remain available in repository history.

License

Derived from facebook/mms-1b, these model weights are licensed CC-BY-NC-4.0, including the non-commercial restriction. The previous CC-BY-4.0 metadata was incorrect and has been corrected.

Citation

@inproceedings{wang2025geolid,
  title={Geolocation-Aware Robust Spoken Language Identification},
  author={Qingzheng Wang and Hye-jin Shim and Jiancheng Sun and Shinji Watanabe},
  booktitle={2025 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)},
  year={2025},
  organization={IEEE},
  url={https://arxiv.org/abs/2508.17148}
}

Recipe and evaluation results