Papers
arxiv:2609.28339

Beyond Future Prediction: Denoising as Generative Adaptation for Robot Control

Published on Sep 23
· Submitted by
Zanyi Wang
on Oct 5
Authors:
,
,
,

Abstract

NowWAM couples generative diffusion objectives directly to robot action prediction via co-training on current observations, achieving faster and more robust control without requiring future visual targets.

Pretrained generative Diffusion Transformers (DiTs) capture rich pixel-level visual and language-conditioned structure through large-scale image and video generation training. A growing line of robot policies builds on this generative prior, but how it should be transferred to control remains unclear, and existing approaches commonly instantiate this transfer through future visual prediction. We ask a more basic question: what a pretrained generative DiT actually contributes to action learning, and how this prior should be adapted for control. We introduce NowWAM, a future-target-free co-training formulation that denoises the current observation and predicts robot actions from the same visual stream, directly coupling the native generative objective to the action-facing representation across the denoising trajectory. Under matched controlled settings, past and future visual targets perform comparably, while restricting training to the clean endpoint substantially reduces robustness, suggesting that a separate future target is not essential for generative adaptation, while the denoising trajectory remains an effective interface for control. On LIBERO-Plus, NowWAM reaches 87.7% with FLUX2-Klein, improving over the future-target co-training baseline by 6.1 points while halving training visual tokens (784 to 392) and reducing step time from 2.85 s to 1.63 s, a 1.8x speedup. With the pure text-to-image Z-Image backbone, NowWAM further reaches 87.8%, showing that strong control adaptation is not tied to video generation or image-editing backbones.

Community

Paper author Paper submitter

Does adapting a pretrained generative model for robot control really require predicting the future?

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

Paper author

We release our code now at https://github.com/xmz111/NowWAM

Thank you for sharing NowWAM. Comparing current-frame denoising with temporal prediction objectives helps isolate what generative training contributes to robot control. We discuss this connection in the working revision of our survey, In-Context Learning for Robots: Methods and Applications (https://arxiv.org/abs/2609.36012), to distinguish training-time objectives from predictive and contextual computation at deployment. If this broader discussion is useful for your research, we would appreciate your considering the survey as a reference. Thanks again!

·
Paper author

Thanks for your interest! Could you point us to where NowWAM is discussed in your survey?

NowWAM was publicly released on September 23, before your survey's initial release on September 28. If your survey covers our findings on training-time generative adaptation, please ensure the original work is properly cited.

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2609.28339
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2609.28339 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2609.28339 in a dataset README.md to link it from this page.

Spaces citing this paper 1

Collections including this paper 1