ShukaiGong commited on
Commit
2dd0db9
Β·
verified Β·
1 Parent(s): 757358d

Move model weights under weights/ and document the Cosmos3-Nano base checkpoint

Browse files

policy/ -> weights/policy/, subgoal_planner/ -> weights/subgoal_planner/. README, NOTICE and release_config.json use the new paths.

Files changed (38) hide show
  1. NOTICE +6 -3
  2. README.md +36 -21
  3. release_config.json +3 -2
  4. {policy β†’ weights/policy}/robotwin_c2r/architecture/Qwen3-VL-8B-Instruct.json +0 -0
  5. {policy β†’ weights/policy}/robotwin_c2r/bundle_manifest.json +0 -0
  6. {policy β†’ weights/policy}/robotwin_c2r/complete.json +0 -0
  7. {policy β†’ weights/policy}/robotwin_c2r/config.json +0 -0
  8. {policy β†’ weights/policy}/robotwin_c2r/data_contract.json +0 -0
  9. {policy β†’ weights/policy}/robotwin_c2r/model/.metadata +0 -0
  10. {policy β†’ weights/policy}/robotwin_c2r/model/__0_0.distcp +0 -0
  11. {policy β†’ weights/policy}/robotwin_c2r/model/__0_1.distcp +0 -0
  12. {policy β†’ weights/policy}/robotwin_c2r/model/__0_2.distcp +0 -0
  13. {policy β†’ weights/policy}/robotwin_c2r/model/__0_3.distcp +0 -0
  14. {policy β†’ weights/policy}/robotwin_c2r/model/__0_4.distcp +0 -0
  15. {policy β†’ weights/policy}/robotwin_c2r/model/__0_5.distcp +0 -0
  16. {policy β†’ weights/policy}/robotwin_c2r/model/__0_6.distcp +0 -0
  17. {policy β†’ weights/policy}/robotwin_c2r/model/__0_7.distcp +0 -0
  18. {policy β†’ weights/policy}/robotwin_c2r/normalizer.json +0 -0
  19. {policy β†’ weights/policy}/robotwin_c2r/recipe_config.json +0 -0
  20. {policy β†’ weights/policy}/robotwin_c2r/text_tokenizer/chat_template.json +0 -0
  21. {policy β†’ weights/policy}/robotwin_c2r/text_tokenizer/config.json +0 -0
  22. {policy β†’ weights/policy}/robotwin_c2r/text_tokenizer/merges.txt +0 -0
  23. {policy β†’ weights/policy}/robotwin_c2r/text_tokenizer/preprocessor_config.json +0 -0
  24. {policy β†’ weights/policy}/robotwin_c2r/text_tokenizer/tokenizer.json +0 -0
  25. {policy β†’ weights/policy}/robotwin_c2r/text_tokenizer/tokenizer_config.json +0 -0
  26. {policy β†’ weights/policy}/robotwin_c2r/text_tokenizer/video_preprocessor_config.json +0 -0
  27. {policy β†’ weights/policy}/robotwin_c2r/text_tokenizer/vocab.json +0 -0
  28. {policy β†’ weights/policy}/robotwin_c2r/vae/Wan2.2_VAE.pth +0 -0
  29. {subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/inference_config.json +0 -0
  30. {subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/.metadata +0 -0
  31. {subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/__0_0.distcp +0 -0
  32. {subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/__1_0.distcp +0 -0
  33. {subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/__2_0.distcp +0 -0
  34. {subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/__3_0.distcp +0 -0
  35. {subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/__4_0.distcp +0 -0
  36. {subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/__5_0.distcp +0 -0
  37. {subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/__6_0.distcp +0 -0
  38. {subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/__7_0.distcp +0 -0
NOTICE CHANGED
@@ -2,13 +2,16 @@ ViGAR model materials
2
 
3
  This repository contains the ViGAR action-policy and subgoal-planner DCP
4
  checkpoints, along with the policy bundle's VAE, tokenizer and configuration
5
- assets. ViGAR provides fine-tuned RoboTwin checkpoints and portable inference
6
- bundle metadata.
7
 
8
  Cosmos3-Nano
9
  Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
10
  The policy and subgoal-planner checkpoints derive from Cosmos3-Nano model
11
- materials. These materials are distributed under OpenMDW-1.1; see LICENSE.
 
 
 
12
  Upstream: https://huggingface.co/nvidia/Cosmos3-Nano
13
 
14
  Wan2.2 VAE
 
2
 
3
  This repository contains the ViGAR action-policy and subgoal-planner DCP
4
  checkpoints, along with the policy bundle's VAE, tokenizer and configuration
5
+ assets, and a DCP conversion of the Cosmos3-Nano base checkpoint. ViGAR provides
6
+ fine-tuned RoboTwin checkpoints and portable inference bundle metadata.
7
 
8
  Cosmos3-Nano
9
  Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
10
  The policy and subgoal-planner checkpoints derive from Cosmos3-Nano model
11
+ materials. weights/Cosmos3-Nano-dcp is Cosmos3-Nano (revision
12
+ fea6e03ac3d7884b4105ed8ee79fc480fca70965) converted to PyTorch Distributed
13
+ Checkpoint format without other changes. These materials are distributed under
14
+ OpenMDW-1.1; see LICENSE.
15
  Upstream: https://huggingface.co/nvidia/Cosmos3-Nano
16
 
17
  Wan2.2 VAE
README.md CHANGED
@@ -12,19 +12,26 @@ tags:
12
 
13
  # ViGAR weights
14
 
15
- Selected RoboTwin inference weights for [ViGAR](https://github.com/DAGroup-PKU/ViGAR):
16
- an action policy and an image subgoal planner.
 
17
 
18
  | Component | Folder | Selection |
19
  |---|---|---|
20
- | Action policy | `policy/robotwin_c2r` | three views, native 49-D, 50k EMA |
21
- | Subgoal planner | `subgoal_planner/robotwin_subgoal_planner` | regular weights |
 
 
22
 
23
- The weights use PyTorch Distributed Checkpoint (DCP). Download all `.distcp`
24
- shards and `.metadata`. See the [installation and evaluation guide](https://github.com/DAGroup-PKU/ViGAR#installation) for the runtime setup.
 
 
25
 
26
  ## Download
27
 
 
 
28
  ```bash
29
  hf download DAGroup-PKU/ViGAR --local-dir /path/to/vigar-weights
30
  ```
@@ -32,32 +39,40 @@ hf download DAGroup-PKU/ViGAR --local-dir /path/to/vigar-weights
32
  Then set in the code repo's `.env`:
33
 
34
  ```bash
35
- VIGAR_POLICY_CHECKPOINT=/path/to/vigar-weights/policy/robotwin_c2r
36
- SUBGOAL_PLANNER_CHECKPOINT=/path/to/vigar-weights/subgoal_planner/robotwin_subgoal_planner
37
- WAN_VAE_PATH=/path/to/vigar-weights/policy/robotwin_c2r/vae/Wan2.2_VAE.pth
38
- QWEN_TOKENIZER_PATH=/path/to/vigar-weights/policy/robotwin_c2r/text_tokenizer
 
 
 
39
  ```
40
 
 
 
 
 
 
41
  The policy bundle carries its architecture config, tokenizer, Wan VAE, recipe and
42
- normalizer. The planner can reuse that bundle's tokenizer/VAE as described in its
43
  `inference_config.json`. Inference uses current-observation Strict Sync β€” policy 10
44
  steps / guidance 1 / shift 2 (predict 48, execute 32); planner 35 steps / guidance
45
  2.5 / shift 5 at RGB 384Γ—320.
46
 
 
 
 
 
 
 
 
 
47
  ## Policy training data
48
 
49
  The [segmented dataset](data/robotwin_segmented_2500) contains 2,500 RoboTwin
50
  training episodes: 19 multi-stage tasks and 31 final-goal tasks, with 3,691 selected
51
- generated goals. Download it from this repository:
52
-
53
- ```bash
54
- hf download DAGroup-PKU/ViGAR --include "data/robotwin_segmented_2500/**" --local-dir /path/to/vigar-weights
55
- ```
56
-
57
- Use `data/robotwin_segmented_2500/dataset.json` as `VIGAR_DATASET_MANIFEST` and
58
- `data/robotwin_segmented_2500/goal_cache` as `VIGAR_GENERATED_GOAL_CACHE`, with the
59
- absolute paths of your download. Dataset licensing and attribution are in its
60
- `LICENSE` and `NOTICE` files.
61
 
62
  ## License
63
 
 
12
 
13
  # ViGAR weights
14
 
15
+ RoboTwin weights and policy training data for [ViGAR](https://github.com/DAGroup-PKU/ViGAR):
16
+ an action policy, an image subgoal planner, the Cosmos3-Nano base checkpoint used
17
+ to train the policy, and the segmented training dataset.
18
 
19
  | Component | Folder | Selection |
20
  |---|---|---|
21
+ | Action policy | `weights/policy/robotwin_c2r` | three views, native 49-D, 50k EMA |
22
+ | Subgoal planner | `weights/subgoal_planner/robotwin_subgoal_planner` | regular weights |
23
+ | Base checkpoint | `weights/Cosmos3-Nano-dcp` | Cosmos3-Nano, policy training start point |
24
+ | Policy training data | `data/robotwin_segmented_2500` | 2,500 episodes, 3,691 goals |
25
 
26
+ The weights use PyTorch Distributed Checkpoint (DCP). Each checkpoint `model/`
27
+ folder holds its `.distcp` shards and `.metadata`. See the
28
+ [installation and evaluation guide](https://github.com/DAGroup-PKU/ViGAR#installation)
29
+ for the runtime setup.
30
 
31
  ## Download
32
 
33
+ Download the whole repository (about 96 GB) once:
34
+
35
  ```bash
36
  hf download DAGroup-PKU/ViGAR --local-dir /path/to/vigar-weights
37
  ```
 
39
  Then set in the code repo's `.env`:
40
 
41
  ```bash
42
+ VIGAR_POLICY_CHECKPOINT=/path/to/vigar-weights/weights/policy/robotwin_c2r
43
+ SUBGOAL_PLANNER_CHECKPOINT=/path/to/vigar-weights/weights/subgoal_planner/robotwin_subgoal_planner
44
+ WAN_VAE_PATH=/path/to/vigar-weights/weights/policy/robotwin_c2r/vae/Wan2.2_VAE.pth
45
+ QWEN_TOKENIZER_PATH=/path/to/vigar-weights/weights/policy/robotwin_c2r/text_tokenizer
46
+ BASE_CHECKPOINT_PATH=/path/to/vigar-weights/weights/Cosmos3-Nano-dcp
47
+ VIGAR_DATASET_MANIFEST=/path/to/vigar-weights/data/robotwin_segmented_2500/dataset.json
48
+ VIGAR_GENERATED_GOAL_CACHE=/path/to/vigar-weights/data/robotwin_segmented_2500/goal_cache
49
  ```
50
 
51
+ Evaluation uses the policy and planner. Policy training uses the base checkpoint,
52
+ the dataset and the policy bundle's tokenizer and Wan VAE.
53
+
54
+ ## Policy and planner
55
+
56
  The policy bundle carries its architecture config, tokenizer, Wan VAE, recipe and
57
+ normalizer. The planner reuses that bundle's tokenizer/VAE as described in its
58
  `inference_config.json`. Inference uses current-observation Strict Sync β€” policy 10
59
  steps / guidance 1 / shift 2 (predict 48, execute 32); planner 35 steps / guidance
60
  2.5 / shift 5 at RGB 384Γ—320.
61
 
62
+ ## Base checkpoint
63
+
64
+ `weights/Cosmos3-Nano-dcp` is [nvidia/Cosmos3-Nano](https://huggingface.co/nvidia/Cosmos3-Nano)
65
+ (revision `fea6e03ac3d7884b4105ed8ee79fc480fca70965`) converted to DCP with
66
+ `cosmos_framework.scripts.convert_model_to_dcp`. Its `model/` folder has six shards
67
+ of about 5 GB each. Policy training loads it through `BASE_CHECKPOINT_PATH`; it is
68
+ not needed for evaluation.
69
+
70
  ## Policy training data
71
 
72
  The [segmented dataset](data/robotwin_segmented_2500) contains 2,500 RoboTwin
73
  training episodes: 19 multi-stage tasks and 31 final-goal tasks, with 3,691 selected
74
+ generated goals. Dataset licensing and attribution are in its `LICENSE` and
75
+ `NOTICE` files.
 
 
 
 
 
 
 
 
76
 
77
  ## License
78
 
release_config.json CHANGED
@@ -1,9 +1,10 @@
1
  {
2
  "repository": "DAGroup-PKU/ViGAR",
3
- "policy": "policy/robotwin_c2r",
4
  "policy_weights": "ema",
5
  "policy_name": "robotwin_c2r",
6
- "subgoal_planner": "subgoal_planner/robotwin_subgoal_planner",
7
  "subgoal_planner_weights": "regular",
 
8
  "policy_dataset": "data/robotwin_segmented_2500"
9
  }
 
1
  {
2
  "repository": "DAGroup-PKU/ViGAR",
3
+ "policy": "weights/policy/robotwin_c2r",
4
  "policy_weights": "ema",
5
  "policy_name": "robotwin_c2r",
6
+ "subgoal_planner": "weights/subgoal_planner/robotwin_subgoal_planner",
7
  "subgoal_planner_weights": "regular",
8
+ "base_checkpoint": "weights/Cosmos3-Nano-dcp",
9
  "policy_dataset": "data/robotwin_segmented_2500"
10
  }
{policy β†’ weights/policy}/robotwin_c2r/architecture/Qwen3-VL-8B-Instruct.json RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/bundle_manifest.json RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/complete.json RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/config.json RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/data_contract.json RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/model/.metadata RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/model/__0_0.distcp RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/model/__0_1.distcp RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/model/__0_2.distcp RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/model/__0_3.distcp RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/model/__0_4.distcp RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/model/__0_5.distcp RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/model/__0_6.distcp RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/model/__0_7.distcp RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/normalizer.json RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/recipe_config.json RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/text_tokenizer/chat_template.json RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/text_tokenizer/config.json RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/text_tokenizer/merges.txt RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/text_tokenizer/preprocessor_config.json RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/text_tokenizer/tokenizer.json RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/text_tokenizer/tokenizer_config.json RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/text_tokenizer/video_preprocessor_config.json RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/text_tokenizer/vocab.json RENAMED
File without changes
{policy β†’ weights/policy}/robotwin_c2r/vae/Wan2.2_VAE.pth RENAMED
File without changes
{subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/inference_config.json RENAMED
File without changes
{subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/.metadata RENAMED
File without changes
{subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/__0_0.distcp RENAMED
File without changes
{subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/__1_0.distcp RENAMED
File without changes
{subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/__2_0.distcp RENAMED
File without changes
{subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/__3_0.distcp RENAMED
File without changes
{subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/__4_0.distcp RENAMED
File without changes
{subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/__5_0.distcp RENAMED
File without changes
{subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/__6_0.distcp RENAMED
File without changes
{subgoal_planner β†’ weights/subgoal_planner}/robotwin_subgoal_planner/model/__7_0.distcp RENAMED
File without changes