--- license: other license_name: openmdw-1.1 license_link: LICENSE library_name: pytorch tags: - robotics - world-model - long-horizon - in-context-learning - RoboTwin ---

# Rethinking World-Action Model for Compositional and In-Context Robotic Manipulation ViGAR: **Vi**sual **G**oal-conditioned **A**ction **R**easoning Shukai Gong^1* · Xuanran Zhai^2* · Yintianrun Zhang^1* · Ruopeng Cui² · Ye Huang¹ · Yiyang Fu¹ · Dexuan Lyu²
Chaojie Li² · Xinyi Song² · Peiwen Lin² · Chuang Wang² · Mingyuan Jia³ · Yufan Deng¹
Jiaxin Fang³ · Bo Liang¹ · Jiaxin Li¹ · Yuxiang Gao^3† · Hao Liu^2† · Daquan Zhou^1† ¹ Peking University ² AgiBot ³ CocoMatrix ^* Equal contribution ^† Corresponding author [![Project Page](https://img.shields.io/badge/Project-Page-2ea44f?logo=googlechrome&logoColor=white)](https://dagroup-pku.github.io/ViGAR/) [![arXiv](https://img.shields.io/badge/arXiv-2610.02368-b31b1b.svg?logo=arxiv&logoColor=white)](https://arxiv.org/abs/2610.02368) [![HF Daily Paper](https://img.shields.io/badge/🤗%20Hugging%20Face-Daily%20Paper-yellow)](https://huggingface.co/papers/2610.02368) [![GitHub](https://img.shields.io/badge/GitHub-Code-181717?logo=github&logoColor=white)](https://github.com/DAGroup-PKU/ViGAR) [![Python](https://img.shields.io/badge/Python-3.13%20%7C%203.10-3776AB?logo=python&logoColor=white)](https://github.com/DAGroup-PKU/ViGAR/blob/main/docs/installation.md) [![PyTorch](https://img.shields.io/badge/PyTorch-2.10-EE4C2C?logo=pytorch&logoColor=white)](https://github.com/DAGroup-PKU/ViGAR/blob/main/docs/installation.md) [![License: OpenMDW-1.1](https://img.shields.io/badge/License-OpenMDW--1.1-blue.svg)](LICENSE)

![ViGAR teaser: task-diverse and long-horizon data train a subgoal planner and a world-action policy, enabling long-horizon manipulation by subtask decomposition and in-context learning on unseen tasks](assets/teaser.jpg) ## 🔥 News - **2026.10.11**: ViGAR ranks #3 on the [RoboTwin 2.0 Leaderboard](https://robotwin-platform.github.io/leaderboard). - **2026.10.05**: ViGAR is released on [arXiv](https://arxiv.org/abs/2610.02368). ## 📑 Todo List - [x] Evaluation code and checkpoints on **RoboTwin**. - [x] Training code and data on **RoboTwin**. - [ ] 900-hour real-robot dataset with subtask annotations. ## 🎬 Demo

## ⚙️ Getting Started ```bash git clone https://github.com/DAGroup-PKU/ViGAR.git cd ViGAR ``` | Step | Guide | | :--- | :--- | | Set up the environments | [Installation](https://github.com/DAGroup-PKU/ViGAR/blob/main/docs/installation.md) | | Download weights and data | [Download](https://github.com/DAGroup-PKU/ViGAR/blob/main/docs/download.md) | | Evaluate on Simulation Benchmark | [Evaluation](https://github.com/DAGroup-PKU/ViGAR/blob/main/docs/evaluation.md) | | Train the policies and subgoal planners in ViGAR | [Training](https://github.com/DAGroup-PKU/ViGAR/blob/main/docs/training.md) | ## 💪 Repository ```text vigar/ Policy model, training and serving subgoal_planner/ Planner runtime and training implementation simulation/ RoboTwin tasks and simulator setup data/ Annotation, goal generation and action conversion evaluation/ Policy/planner services and closed-loop evaluation scripts/ Installation and launch commands configs/ Task and inference settings docs/ Installation, download, evaluation and training guides ``` The policy uses VeOmni with a bundled Cosmos runtime. Planner processes select their own Cosmos runtime through `PYTHONPATH`; they run separately from policy and simulator processes. ## 🙏 Acknowledgements Built on [NVIDIA Cosmos](https://github.com/NVIDIA/cosmos), [VeOmni](https://github.com/ByteDance-Seed/VeOmni), [RoboTwin](https://github.com/RoboTwin-Platform/RoboTwin), [Wan2.2](https://github.com/Wan-Video/Wan2.2) and [Qwen3-VL](https://github.com/QwenLM/Qwen3-VL). ## ✏️ Citation ```bibtex @article{gong2026rethinking, title={Rethinking World-Action Model for Compositional and In-Context Robotic Manipulation}, author={Gong, Shukai and Zhai, Xuanran and Zhang, Yintianrun and Cui, Ruopeng and Huang, Ye and Fu, Yiyang and Lyu, Dexuan and Li, Chaojie and Song, Xinyi and Lin, Peiwen and others}, journal={arXiv preprint arXiv:2610.02368}, year={2026} } ``` ## License Original project code uses the [MIT license](https://github.com/DAGroup-PKU/ViGAR/blob/main/LICENSE). Bundled components retain their upstream licenses. Cosmos-derived model materials use OpenMDW-1.1.