CoffeeBEEE/robotwin2_pi0_place_shoe_reproduction

Pi_0 — RoboTwin 2.0 Place Shoe (LoRA fine-tune)

Fine-tuned openpi Pi_0 (3.3B, Physical Intelligence) for the place_shoe task of the RoboTwin 2.0 benchmark (dual-arm table-top manipulation), using LoRA on 50 clean simulator demonstrations. This work is part of our reproduction of the RoboTwin 2.0 paper (arXiv 2506.18088): we fine-tune the paper's Pi_0 baseline on place_shoe and evaluate it under the benchmark's protocol (100 rollouts, Hard split only).

⚠️ Checkpoint format: this repository stores an openpi / JAX / Orbax (OCDBT) checkpoint, not a PyTorch .safetensors model. It is loaded natively by the openpi serving path (create_trained_policy). See Reproduction — no conversion is needed or supported for the RoboTwin eval path used here.

Model Details

Property	Value
Base model	pi0_base (openpi-assets, orbax/OCDBT format)
Adapter	LoRA fine-tune
Train config	pi0_base_aloha_full_sim_arx-x5_seed_0
Training data	RoboTwin 2.0 place_shoe, demo_clean — 50 episodes
Training steps	30,000
Robot / env	ALOHA (agilex) dual arm — env_cfg_type=aloha_agilex, action_type=joint
Checkpoint step	29999 (~5.8 GB, bf16 full model incl. LoRA)
Norm stats asset id	arx_x5_sim
Framework	JAX + Orbax (OCDBT); served via XPolicyLab policy/Pi_0 adapter

The training run produced checkpoints at steps 17000 / 20000 / 25000 / 29999. This repository contains only the final step 29999 — the weights used for the reported evaluation.

Evaluation

Protocol follows the RoboTwin 2.0 benchmark: 50 demo_clean demos for training, 100 rollouts for evaluation, Easy (demo_clean) / Hard (demo_randomized) reported separately (paper: arXiv 2506.18088).

Setting	Success rate
Hard (demo_randomized), this model	12 / 100 (12%)
Paper baseline — Pi0, Hard (demo_randomized)	6%
Paper baseline — Pi0, Easy (demo_clean)	28%

Eval details for the 12% result: demo_randomized, 100 rollouts, --seed 0 (scene seed sequence starting at 100000), instruction type unseen, expert feasibility check on (expert_check=true, framework default). For the leaderboard's full table, see RoboTwin 2.0 and Table 18 of the paper.

Demos

Training demo — expert demonstration from the demo_clean training data (episode0):

Evaluation demo — a policy rollout from the Hard (demo_randomized) 100-rollout evaluation (episode51):

Reproduction

1. Base model

The base pi0_base weights are required. They are not included in this repository. They must be in openpi's native checkpoint format (orbax / OCDBT shards), as loaded by the training config's CheckpointWeightLoader:

• GCS (canonical): gs://openpi-assets/checkpoints/pi0_base/params — downloaded automatically by openpi on first run (see openpi/shared/download.py).
• ModelScope mirror (China-friendly): masheng/pi0-fine-Tuned-Models → pi0_base.zip, verified byte-identical to the GCS copy (CRC32 24/24).

⚠️ Do not use lerobot/pi0_base on Hugging Face as the base for this training path: it is the LeRobot framework's separate checkpoint (model.safetensors, 14 GB, PyTorch format), not in openpi's orbax format — openpi's CheckpointWeightLoader cannot load it directly.

2. Checkpoint layout

Place this repo's contents so the eval resolver finds a step directory with params/ + assets/. If the repo root already contains params/ and assets/, it is used directly:

XPolicyLab/policy/Pi_0/checkpoints/
└── RoboTwin-pi0_place_shoe-aloha_agilex-joint-0/
    └── 29999/                     ← this repository (contents at repo root also work)
        ├── params/                # OCDBT weight shards
        ├── assets/arx_x5_sim/     # normalization stats (required)
        ├── train_state/           # optimizer state (not needed for eval)
        └── _CHECKPOINT_METADATA

Note: deploy.yml in the adapter has checkpoint_num: 59999 as a template value. The XPolicyLab resolver (_resolve_pi_model_root) auto-selects the highest step directory when no exact match exists — here it resolves to 29999. No edit is required to reproduce the reported result.

3. Run the RoboTwin eval (Hard)

# in the XPolicyLab repo
cd $XPL_ROOT
bash policy/Pi_0/eval.sh \
  RoboTwin place_shoe demo_randomized \
  RoboTwin-pi0_place_shoe-aloha_agilex-joint-0 joint 0 0

This starts the openpi websocket policy server, runs 100 rollouts on demo_randomized, and writes per-episode videos + _result.txt. See the adapter README.md for the exact argument order.

Limitations

• Fine-tuned on a single task (place_shoe) and a single embodiment (ALOHA); not generalizable to other tasks without further training.
• Eval is reported on the Hard split only (demo_randomized, 12%). The Easy (demo_clean) number was not re-measured for this checkpoint.
• 12% is a single 100-rollout draw (--seed 0); 95% CI ≈ [5.6%, 18.4%] — directionally above the paper's Pi0 Hard baseline (6%), not a statistically guaranteed margin.
• Expert feasibility check (expert_check=true) filters scenes the scripted expert cannot solve, consistent with the framework default.

Citation

@misc{robotwin2,
  title={RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization},
  author={Chen, Tianxing and others},
  year={2025},
  eprint={2506.18088},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2506.18088}
}

Base model: Pi_0 (Physical Intelligence, openpi).