OzzyGT/Kimodo_SOMA_RP_v1_2

Kimodo SOMA RP v1.2

A four-second running motion, seed 42, 100 steps, at normal playback speed. The cameras follow the original root motion.

A Diffusers conversion of NVIDIA's Kimodo SOMA RP v1.2 for text-to-motion generation, including its Llama 3 / LLM2Vec text encoder. Built with Meta Llama 3.

For model details and prompting, see the original model card and Kimodo repository.

Usage

Install PyTorch for your GPU, then:

pip install diffusers==0.41.0 transformers==5.20.2 peft==0.21.2 "accelerate>=1.15" "safetensors>=0.8" numpy

This example uses the separate Kimodo modular blocks. After publishing both repositories, the example below loads the separate OzzyGT/Kimodo_SOMA_Modular blocks. A downloaded blocks directory can also be used.

import numpy as np
import torch
from diffusers import ModularPipelineBlocks

blocks_path = "OzzyGT/Kimodo_SOMA_Modular"
model_id = "OzzyGT/Kimodo_SOMA_RP_v1_2"

blocks = ModularPipelineBlocks.from_pretrained(
    blocks_path,
    trust_remote_code=True,
    components_repo=model_id,
    trust_components_code=True,
)
pipe = blocks.init_pipeline()
pipe.load_components(dtype={"text_encoder": torch.bfloat16, "default": torch.float32})
pipe.to("cuda")

motion = pipe(
    prompt="A person runs forward at a steady pace.",
    num_frames=120,
    seed=42,
    num_inference_steps=100,
    output="motion",
)
np.savez_compressed(
    "motion.npz",
    **{key: value[0].cpu().numpy() for key, value in motion.items()},
)

This generates four seconds at 30 FPS. Output uses meters, +Y up, initial +Z forward, and the SOMA77 skeleton, with root motion preserved. Change seed for another variation. Negative prompts are not supported.

Licenses

• Motion weights: NVIDIA Open Model License, from nvidia/Kimodo-SOMA-RP-v1.2.
• Text encoder: Meta Llama 3 Community License and Acceptable Use Policy. Based on Meta Llama 3 8B Instruct with the LLM2Vec MNTP and supervised adapters.
• Custom model code: Apache-2.0. Adapted LLM2Vec code retains its MIT license. See NOTICE for attribution.