Learning Multiple Utterance-Level Attribute Representations with a Unified Speech Encoder
Paper • 2603.08312 • Published
How to use LIA-AvignonUniversity/SENSE-2ATTR with speechbrain:
# interface not specified in config.json
SENSE-2ATTR is a unified speech encoder that jointly learns multiple utterance-level attribute representations from a single speech signal.
This version models two attributes:
| Attribute | Teacher model | Dimension |
|---|---|---|
| Semantic | BGE-M3 | 1024 |
| Speaker | ECAPA-TDNN | 192 |
pip install speechbrain
from speechbrain.inference.interfaces import foreign_class
model = foreign_class(
source="LIA-AvignonUniversity/SENSE-2ATTR",
pymodule_file="custom.py",
classname="SENSEEncoder",
run_opts={"device": "cuda:0"},
)
Extract the representations from an audio file:
outputs = model.encode_file("example.wav")
semantic_embedding = outputs["sem"]
# torch.Size([1, 1024])
speaker_embedding = outputs["spkr"]
# torch.Size([1, 192])
Learning Multiple Utterance-Level Attribute Representations with a Unified Speech Encoder
Maryem Bouziane, Salima Mdhaffar, Yannick Estève
https://arxiv.org/abs/2603.08312
If you use this model, please cite:
@misc{bouziane2026learning,
title={Learning Multiple Utterance-Level Attribute Representations with a Unified Speech Encoder},
author={Maryem Bouziane and Salima Mdhaffar and Yannick Est{\`e}ve},
year={2026},
eprint={2603.08312},
archivePrefix={arXiv}
}
Laboratoire Informatique d'Avignon (LIA)
Avignon Université, France