Hugging Face's logo

acul3
/
Qwen3-TTS-1.7B-Base-ExecuTorch

Text-to-Speech
ExecuTorch
tts
voice-cloning
on-device
android
xnnpack
int8
qwen3
speech-synthesis
Model card Files
xet
 Community

Qwen3-TTS Voice Clone — ExecuTorch (Android-ready)

On-device text-to-speech with voice cloning, converted from Qwen3-TTS-1.7B-Base to ExecuTorch .pte format for mobile/edge deployment.

1.9B parameter end-to-end TTS — clone any voice from a short audio sample and synthesize speech entirely on-device. No cloud, no internet needed.

Models

INT8 Quantized (⭐ Recommended for on-device)

Module Size Description
speaker_encoder_int8.pte 46 MB Extract speaker identity from reference audio
talker_int8.pte 1.4 GB Main autoregressive LM (generates audio codec tokens)
code_predictor_int8.pte 78 MB Multi-codebook prediction (15 additional codebooks)
vocoder_int8.pte 301 MB Neural vocoder (codec tokens → PCM waveform)
Total 1.8 GB Fits on 8GB+ phones

FP32 Unquantized

Module Size
speaker_encoder.pte 46 MB
talker_prefill.pte 5.3 GB
code_predictor.pte 309 MB
vocoder.pte 436 MB

Auxiliary Files

File Description
talker_embeddings.pt Text + codec embedding tables (loaded in Python orchestrator)
code_predictor_extras.pt Code predictor embedding + projection weights

Architecture 

Qwen3-TTS Voice Clone Pipeline (1.9B params total)

Input: text + reference audio (3-5s voice sample)
                    │
        ┌───────────┴───────────┐
        ▼                       ▼
  Speaker Encoder (12M)    Speech Tokenizer
  TDNN → AttPool → FC     (encode ref audio
  ref_audio → x_vector     to codec codes)
  [1, 2048]                [T, 16]
        │                       │
        └───────┬───────────────┘
                ▼
         Talker LM (1.7B)
         Qwen3, 28 layers, GQA 16/8
         dim=2048, audio vocab=3072
         Autoregressive codec generation
                │
                ▼
         Code Predictor (175M)
         Predict 15 additional codebooks
         per token (residual VQ)
                │
                ▼
         Vocoder (154M)
         Codec tokens → 24kHz PCM audio
                │
                ▼
         Output: speech waveform (.wav)

Component Details

Component Params Architecture Input → Output
Speaker Encoder 12M TDNN + Attentive Stats Pooling mel spectrogram → x_vector [1, 2048]
Talker (Main LM) 1,727M Qwen3, 28 layers, GQA 16/8 heads, dim 2048 text + speaker emb → codec tokens (vocab 3072)
Text Projection 8M MLP text hidden → audio hidden dim
Codec Head 6M Linear hidden states → first codebook logits
Code Predictor 175M Small LM + 15 heads main LM output → codebooks 2-16
Vocoder 154M Qwen3TTSTokenizerV2Model [16, T] codes → 24kHz waveform

How It Works

Voice Clone Pipeline 

1. ref_audio (24kHz, 3-5s) → mel_spectrogram → speaker_encoder.pte → x_vector [1, 2048]
2. ref_audio → speech_tokenizer.encode() → ref_codes [T, 16]  (runs on CPU, not exported)
3. text → Qwen2 tokenizer → input_ids
4. Embed: text_embedding(input_ids) + codec_embedding(ref_codes) + x_vector → inputs_embeds
5. talker.pte(inputs_embeds, kv_cache, ...) → codec logits  (autoregressive loop)
6. code_predictor.pte(hidden_states) → codebooks 2-16  (per step)
7. All codec codes → vocoder.pte → 24kHz PCM waveform

Token Format

The talker uses an interleaved text+audio token sequence:

[BOS] [text tokens...] [speaker x-vector] [ref audio codes...] [SEP] [generated audio codes...]

Key Parameters

Parameter Value
Audio sample rate 24,000 Hz
Codec frame rate 12.5 Hz (80ms per frame)
Codebooks 16 (1 from talker + 15 from code predictor)
Audio vocab size 3,072
Text vocab size 151,936 (Qwen2 tokenizer)
Max sequence length 2,048 tokens
Speaker embedding dim 2,048

Quick Start — Python 

from huggingface_hub import hf_hub_download
from executorch.runtime import Runtime
import torch
import numpy as np

REPO = "acul3/Qwen3-TTS-1.7B-Base-ExecuTorch"

# Download INT8 models
spk_path = hf_hub_download(REPO, "speaker_encoder_int8.pte")
talker_path = hf_hub_download(REPO, "talker_int8.pte")
cp_path = hf_hub_download(REPO, "code_predictor_int8.pte")
voc_path = hf_hub_download(REPO, "vocoder_int8.pte")
emb_path = hf_hub_download(REPO, "talker_embeddings.pt")
cp_extras_path = hf_hub_download(REPO, "code_predictor_extras.pt")

# Load ExecuTorch runtime
runtime = Runtime.get()
speaker_enc = runtime.load_program(spk_path).load_method("forward")
vocoder = runtime.load_program(voc_path).load_method("forward")

# Load embeddings (used in Python orchestration)
embeddings = torch.load(emb_path, weights_only=True)
cp_extras = torch.load(cp_extras_path, weights_only=True)

# For full generation, see scripts/test_e2e.py

Quick Start — Android (Kotlin) 

import org.pytorch.executorch.Module

// Load all 4 modules
val speakerEnc = Module.load("speaker_encoder_int8.pte")
val talker = Module.load("talker_int8.pte")
val codePred = Module.load("code_predictor_int8.pte")
val vocoder = Module.load("vocoder_int8.pte")

// Pipeline:
// 1. Process ref audio → mel → speaker_enc.forward() → x_vector
// 2. Build input embeddings (text + speaker + ref codes)
// 3. Autoregressive loop: talker.forward() → codec token → code_pred.forward() → all codebooks
// 4. vocoder.forward(all_codes) → PCM audio
// See scripts/ for full implementation details

Validation Results

Component Method Cosine Similarity
Speaker Encoder .pte vs PyTorch 0.965*
Talker Wrapper vs Original 1.000 ✅
Vocoder .pte vs PyTorch 1.000 ✅
Code Predictor .pte validated ✅

*Speaker encoder: 0.965 due to mel padding for fixed-size export. With matching sizes: 1.000.

INT8 quantization produces valid, intelligible speech — tested with full pipeline generation.

Export Details

Property Value
ExecuTorch 1.1.0
Backend XNNPACK (CPU, cross-platform)
Quantization torchao INT8 weight-only (per-channel, instant, no calibration)
Source model Qwen/Qwen3-TTS-1.7B-Base
Max sequence length 2,048
Speaker encoder input Fixed 469 mel frames (~3.8s at 24kHz)

Export Challenges Solved

  1. Conv1d padding="same" — Replaced with explicit F.pad() + Conv1d(padding=0) (ExecuTorch doesn't support padding="same")
  2. DynamicCache — Replaced with static KV cache tensors as model inputs/outputs
  3. MROPE (Multi-Resolution RoPE) — Simplified: all 3 dimensions share identical position_ids for TTS
  4. Stride-0 tensors — Used .repeat() instead of .expand() for ExecuTorch compatibility
  5. Vocoder dynamic chunking — Bypassed chunked_decode with fixed code length

Scripts

Script Description
scripts/analyze_model.py Deep architecture analysis + shape tracing
scripts/export_speaker_encoder.py Speaker encoder surgery + .pte export
scripts/export_talker.py Main talker LM surgery + .pte export
scripts/export_code_predictor.py Code predictor surgery + .pte export
scripts/export_vocoder.py Vocoder surgery + .pte export
scripts/quantize_all.py INT8 weight-only quantization of all modules
scripts/test_e2e.py End-to-end validation

Hardware Requirements

On-device inference (INT8)

  • RAM: 8 GB minimum (models use ~1.8 GB + KV cache + audio buffers)
  • Storage: 1.8 GB for all 4 model files + extras
  • CPU: ARM64 (Android) or x86_64

Export/development

  • 64 GB+ unified memory recommended (Jetson AGX Orin or similar)
  • Python 3.10+, PyTorch 2.10+, ExecuTorch 1.1.0, torchao

Reproduce 

# Setup
git clone https://huggingface.co/acul3/Qwen3-TTS-1.7B-Base-ExecuTorch
cd Qwen3-TTS-1.7B-Base-ExecuTorch
pip install executorch torchao qwen-tts

# Export all modules
python scripts/export_speaker_encoder.py
python scripts/export_talker.py
python scripts/export_code_predictor.py
python scripts/export_vocoder.py

# Quantize
python scripts/quantize_all.py

# Validate
python scripts/test_e2e.py

License

Apache 2.0 (same as source model)

Citation 

@misc{qwen3tts_executorch_2026,
  title = {Qwen3-TTS-1.7B-Base-ExecuTorch: On-Device Voice Clone TTS},
  author = {Samsul Rahmadani},
  year = {2026},
  url = {https://huggingface.co/acul3/Qwen3-TTS-1.7B-Base-ExecuTorch},
  note = {Converted from Qwen/Qwen3-TTS-1.7B-Base}
}

Acknowledgments

Downloads last month
54
Inference Providers NEW
Text-to-Speech
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support