kesai-labs/py123d_garage_pretrained_checkpoints

Code | Docs | Research logs | Leaderboard | Blog

Overview

Pretrained driving policies of py123d_garage, the reference toolkit of the Alpasim E2E Challenge 2026. All belong to release v0.2.0 and were trained on Physical AI AV, NuRec and nuPlan.

Policy	Description	PAI	nuPlan
ltf-streaming-joint64-68M	Camera-only latent TransFuser, streaming ResNet-34 backbone that encodes each camera frame once; joint planner of waypoints, actions and a 64 m path	71.3 ± 3.2	90.3 ± 0.6
ltf-base-joint64-68M	Camera-only latent TransFuser, ResNet-34 backbone; joint planner of waypoints, actions and a 64 m path	76.0 ± 0.8	91.4 ± 1.0
ltf-base-joint32-68M*	Camera-only latent TransFuser, ResNet-34 backbone; joint planner of waypoints, actions and a 32 m path	74.9 ± 0.9	90.3 ± 0.5
ltf-base-waypoints-68M*	Camera-only latent TransFuser, ResNet-34 backbone; waypoint-only flow planner	75.5 ± 1.8	89.6 ± 0.5

Scores: local AlpaSim closed loop, 0 to 100, mean ± std over three seeds, on the 441 NuRec scenes of the Physical AI track and the 1485 scenes of the nuPlan track. Setting: take-over after 1.7 s on NuRec and 0.5 s on nuPlan, 2 Hz replanning, no route memory or extrapolation, NuRec harmonizer on, AlpaSim commit e121e59. Each policy's folder holds the per-scene results under results/.

* submitted to the AlpaSim E2E Challenge 2026 leaderboard; the policy's README lists the leaderboard's result.

License and citation

The checkpoints are released under CC BY-NC-SA 4.0, the license of nuPlan, so they are for non-commercial use only. Their training data keeps its own terms: the nuPlan Terms of Use and the NVIDIA Autonomous Vehicle Dataset License of Physical AI AV and NuRec. The code is released under Apache 2.0.

If you use the checkpoints, please cite:

@article{Dauner2026ARXIV,
  title={123D: Unifying Multi-Modal Autonomous Driving Data at Scale},
  author={Dauner, Daniel and Charraut, Valentin and Berle, Bastian and Li, Tianyu and Nguyen, Long and Wang, Jiabao and Jing, Changhui and Igl, Maximilian and Caesar, Holger and Ivanovic, Boris and Geiger, Andreas and Chitta, Kashyap},
  journal={arXiv preprint arXiv:2605.08084},
  year={2026}
}

@inproceedings{Nguyen2026CVPR,
  title={LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Driving},
  author={Long Nguyen and Micha Fauth and Bernhard Jaeger and Daniel Dauner and Maximilian Igl and Andreas Geiger and Kashyap Chitta},
  booktitle={Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2026}
}

Acknowledgements

Our training recipe draws on ideas from many works, above all Alpamayo, the 2025 BEHAVIOR Challenge winner and Qwen-Drive.