Cosmos 3 Nano, full SFT on UMI tossing: control retrain (ctrl_v2)

A second training run of the exact recipe behind Nirvana-AI/umi-toss-cosmos3-nano-full-sft (same data, code, seed 42, sample order, batch 2,024, 2,000 iterations, 4 ร— A100, FSDP-4), trained 2026-11-02 to 2026-11-04 alongside the image-augmentation run Nirvana-AI/umi-toss-cosmos3-nano-aug-v1. It is the control of that ablation: the only difference between the two is the training-time image augmentation. Power-EMA weights of the selected iteration 2,000, bf16.

Selection (val subset, lower is better): composite 0.235 (original run 0.227), uniform ADE 27.2 mm (25.5), release timing MAE 27.4 ms (28.8).

Closed-loop results (Isaac Lab, floating hand, ball in the target basket at the end)

All three Cosmos checkpoints ran the same procedure on the same episodes: the bare policy, then the benchmark's FLUX-equivalent re-tune with weights frozen (CMA-ES knobs + sampler-seed race + validation selection; no earlier add-ons, no residual RL). Counts are of 400 episodes (frozen 100 + held-out 300). Paired exact McNemar tests in results/hillclimb/cosmos_aug_ablation/summary.md of the code repository.

checkpoint bare near (0.92 m) re-tuned near bare far (1.58 m) re-tuned far
original full-sft (iteration 2000) 1 381 0 294
control ctrl-v2 (iteration 2000) 5 304 0 274
+ augmentation aug-v1 (iteration 750) 51 396 0 208
+ augmentation, iteration 2000 (not published) 44 not run 0 not run

Brightness check (re-tuned near add-on, 100 validation episodes, benchmark render then two slightly brighter renders): original 95 / 87 / 90, control 81 / 76 / 74, augmented 99 / 98 / 99.

Through the robot's whole-body controller every checkpoint scores 0โ€“5 of 100. One training run per arm; retraining alone moved the re-tuned near score from 381 to 304, so treat the re-tuned differences with care. The bare near-basket gain (p โ‰ค 3e-9) and the brightness insensitivity are the clear augmentation effects; at the far basket the augmented re-tune is worse.

Loading, licence and provenance are as for the original checkpoint; see its model card. Code: nrvna-ai/umi-tossing-benchmark branch cosmos-aug-ablation (private repository). Nothing was run on a real robot.

Downloads last month
-
Video Preview
loading