README.md

---
license: mit
library_name: transformers
tags:
  - robotics
  - reward-model
  - video-language-model
  - reasoning
  - reinforcement-learning
  - qwen3-vl
  - bf16
pipeline_tag: image-text-to-text
datasets:
  - Philip-MIT/sole_training_data
---

![fig1_v3_padded](/static-proxy?url=https%3A%2F%2Fcdn-uploads.huggingface.co%2Fproduction%2Fuploads%2F6a13185f0f5f7894f043e8d7%2FPO7HMI7QAFyhPiuBRTk9z.png%3C%2Fspan%3E)%3C!----%3E%3C%2Ftd%3E%3C%2Ftr%3E%3Ctr id="L18">

# SOLE-R1-8B

SOLE-R1-8B is a video-language reward reasoning model for robotics. It is designed to estimate task progress from robot video frames and a natural-language task description, producing both per-timestep reasoning traces and scalar progress predictions that can be used as rewards for online robot reinforcement learning.

This model accompanies the paper **“SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot RL”** by Philip Schroeder, Thomas Weng, Karl Schmeckpeper, Eric Rosen, Stephen Hart, and Ondrej Biza.

- Paper: https://arxiv.org/abs/2603.28730
- Project page: https://philip-mit.github.io/sole-r1/
- Code: https://github.com/Philip-MIT/sole-r1-model
- Training data: https://huggingface.co/datasets/Philip-MIT/sole_training_data

## Model Description

SOLE-R1 predicts robot task progress from visual observations. Given a video and a task description, the model outputs a reasoning trace and a scalar progress estimate.

Expected output format:

    <think>reasoning about task progress</think><answer>progress%</answer>

The progress estimate is intended to serve as a dense reward signal for robotic reinforcement learning, especially when manually engineered rewards are unavailable.


## Quick Start

The recommended interface for inference is [RewardGen](https://github.com/Philip-MIT/rewardgen):

    # pip install -U rewardgen

    from rewardgen import generate, video_plot

    # test_videos provided at the github repo: https://github.com/Philip-MIT/rewardgen
    video_paths = [
        "test_videos/robosuite/lift/unsuccessful/robosuite_lift_episode_12_unsuccessful_max_reward_38.mp4"
    ]

    task_description = "Pick up the cube from the table."

    rewards, reasoning_traces = generate(
        model="SOLE-R1",
        task_description=task_description,
        video_paths=video_paths,
        view_type_per_video=["external and wrist"],
        verbose=False,
    )
    print(rewards)
    print(reasoning_traces)

    # Plotting with show_reasoning_traces=True
    output_sole = {"model": "SOLE-R1", "rewards": rewards[0], "reasoning_traces": reasoning_traces[0]}
    video_plot(
        outputs=[output_sole], 
        plot_save_path='model_outputs/sole-r1/robosuite/lift/unsuccessful/robosuite_lift_episode_12_unsuccessful_max_reward_38.mp4', 
        video_path=video_paths[0],
        show_reasoning_traces=True,
        task_description=task_description,
        verbose=False
    )




Optional pre-download:

    from rewardgen.utils.model_utils import get_model_dir

    get_model_dir("sole-r1")

## Input Format

The model is trained to reason over robot task progress using prompts that include:

- A robot task description
- The first timestep progress, typically `0%`
- The previous timestep progress
- Visual observations from the first, previous, and current timesteps
- Multiple camera views when available, such as external and wrist cameras

Example task description:

    Pick up the cube from the table.

## Output Format

The expected output format is:

    <think>[reasoning about visual task progress]</think><answer>[current task progress]%</answer>

Example:

    <think>The gripper has moved closer to the cube but has not yet grasped or lifted it. This indicates incremental progress from the previous timestep.</think><answer>22%</answer>

Downstream systems should parse the numeric value inside `<answer>...</answer>` as the reward/progress estimate.

## Training Data

The model was trained on the [SOLE-R1-8B](https://huggingface.co/Philip-MIT/SOLE-R1-8B)  training dataset.

The dataset contains robot task progress examples with images, prompts, reasoning completions, and progress labels. 

It also includes a diverse collection of general spatial and multi-frame temporal reasoning data (e.g., from SSR-CoT, SpatialVLM, Spot-the-diff, Embodied CoT, RoboVQA, Robo2VLM-Reasoning) to serve as a foundational layer of our training mixture.

The full dataset is approximately 2TB.

Streaming example:

    from datasets import load_dataset

    ds = load_dataset(
        "Philip-MIT/sole_training_data",
        split="train",
        streaming=True,
    )

    for row in ds:
        print(row)
        break

## Citation

BibTeX:

    @misc{schroeder2026soler1,
      title={SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot RL},
      author={Philip Schroeder and Thomas Weng and Karl Schmeckpeper and Eric Rosen and Stephen Hart and Ondrej Biza},
      year={2026},
      eprint={2603.28730},
      archivePrefix={arXiv},
      primaryClass={cs.RO}
    }

## License

This repository is released under the MIT License.