Instructions to use Nirvana-AI/umi-toss-cosmos3-nano-ctrl-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Cosmos
How to use Nirvana-AI/umi-toss-cosmos3-nano-ctrl-v2 with Cosmos:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Cosmos 3 Nano, full SFT on UMI tossing: control retrain (ctrl_v2)
A second training run of the exact recipe behind Nirvana-AI/umi-toss-cosmos3-nano-full-sft (same data, code, seed 42, sample order, batch 2,024, 2,000 iterations, 4 ร A100, FSDP-4), trained 2026-11-02 to 2026-11-04 alongside the image-augmentation run Nirvana-AI/umi-toss-cosmos3-nano-aug-v1. It is the control of that ablation: the only difference between the two is the training-time image augmentation. Power-EMA weights of the selected iteration 2,000, bf16.
Selection (val subset, lower is better): composite 0.235 (original run 0.227), uniform ADE 27.2 mm (25.5), release timing MAE 27.4 ms (28.8).
Closed-loop results (Isaac Lab, floating hand, ball in the target basket at the end)
All three Cosmos checkpoints ran the same procedure on the same episodes: the bare policy, then the benchmark's FLUX-equivalent re-tune with weights frozen (CMA-ES knobs + sampler-seed race + validation selection; no earlier add-ons, no residual RL). Counts are of 400 episodes (frozen 100 + held-out 300). Paired exact McNemar tests in results/hillclimb/cosmos_aug_ablation/summary.md of the code repository.
| checkpoint | bare near (0.92 m) | re-tuned near | bare far (1.58 m) | re-tuned far |
|---|---|---|---|---|
original full-sft (iteration 2000) |
1 | 381 | 0 | 294 |
control ctrl-v2 (iteration 2000) |
5 | 304 | 0 | 274 |
+ augmentation aug-v1 (iteration 750) |
51 | 396 | 0 | 208 |
| + augmentation, iteration 2000 (not published) | 44 | not run | 0 | not run |
Brightness check (re-tuned near add-on, 100 validation episodes, benchmark render then two slightly brighter renders): original 95 / 87 / 90, control 81 / 76 / 74, augmented 99 / 98 / 99.
Through the robot's whole-body controller every checkpoint scores 0โ5 of 100. One training run per arm; retraining alone moved the re-tuned near score from 381 to 304, so treat the re-tuned differences with care. The bare near-basket gain (p โค 3e-9) and the brightness insensitivity are the clear augmentation effects; at the far basket the augmented re-tune is worse.
Loading, licence and provenance are as for the original checkpoint; see its model card. Code: nrvna-ai/umi-tossing-benchmark branch cosmos-aug-ablation (private repository). Nothing was run on a real robot.
- Downloads last month
- -