H3 Avatar

Single-image talking-avatar generation powered by a locked MiniMax-H3 Ref2VA LoRA.

What It Does

H3 Avatar turns one character image plus a short line of dialogue into a talking avatar clip with model-generated speech, synchronized mouth movement, subtle head motion, and controllable scene direction.

The locked product recipe is intentionally simple:

upload one reference image
write the dialogue the character should say
optionally describe the direction/background
generate with the locked H3 Avatar LoRA

Model Details

Item	Value
Base	MiniMax-H3 Ref2VA
Adapter	LoRA
Rank / alpha	`16 / 32`
Target modules	`to_qkv`, `to_out.0`, `linear_1`, `linear_2`
Trainable params	`10,655,744`
Locked checkpoint	`models/locked/h3-avatar-audio-v2-step1000.pt`
Partner adapter	`partner_bundle/adapter_model.safetensors`

Training summary:

481 accepted transcript-conditioned clips
385 train clips
1000 optimizer steps
about 2.6 train-set passes
384x384, 56-frame cache
locked inference default: 30 diffusion steps

Files

partner_bundle/ - clean LoRA adapter export for hosted inference partners.
models/locked/ - locked training checkpoint used by the local/Modal reference path.
modal_h3_avatar.py - Modal reference endpoint.
ui/h3_avatar_ui.py - local H3 Avatar Studio UI.
training/ - cache prep and LoRA training code.
recipes/ - training recipes, including the locked recipe.
evals/ and benchmarks/ - prompts, scorecards, and checkpoint reports.
aq/ and aqit/ - AQ handoff and AQ interpretability notes.

Recommended Inference

{
  "reference_image": "path_or_url_to_image",
  "dialogue": "Hey Chandan, wassup yaar, how are you doing these days?",
  "direction": "The same character faces the camera and speaks with an audible natural voice, expressive synchronized mouth movement, subtle head motion, and a friendly expression.",
  "background": "in a clean studio with soft cinematic lighting",
  "steps": 30
}

For production hosting, keep the H3 base model warm and load this LoRA adapter as the product checkpoint.

Local UI

H3_WARM_ENDPOINT_URL=https://aquin--h3-avatar-warm-generate.modal.run \
python3 ui/h3_avatar_ui.py

Then open:

http://127.0.0.1:8787

Modal Reference Endpoint

modal deploy modal_h3_avatar.py

The Modal path is a reference implementation. Production hosting should use a dedicated inference partner such as Reactor or fal with the base H3 model kept hot.

License

The H3 Avatar adapter, UI code, training recipes, and accompanying project files in this repository are released under the Apache License 2.0.

The adapter is designed to run with MiniMax-H3 Ref2VA. Use of the underlying base model remains subject to the upstream MiniMax-H3 license and any applicable dataset, likeness, and deployment restrictions.

Downloads last month: -