NaP-Control: Navigating Diffusion Prior for Versatile and Fast Character Control

Chia-Wen Chen

ETH Zurich

Video

Abstract

Achieving precise, versatile whole-body character control in physics-based animation remains challenging. Recent diffusion-based policies generate rich and expressive motions but typically rely on gradient-based test-time guidance to satisfy task objectives, which is slow and can reduce robustness. We introduce NaP-Control (Navigating Diffusion Prior for Versatile and Fast Character Control), abbreviated as NaP. Our method uses reinforcement learning to manipulate the latent noise of a task-agnostic diffusion policy prior, steering it toward task-specific behaviors for fast, robust control with high motion fidelity. In contrast to methods that rely solely on offline training, NaP interacts with the environment during training to correct motions and optimize task rewards, improving success rates and enabling adaptation to challenging scenarios. By directly predicting task-optimized diffusion noise, NaP eliminates iterative guidance during denoising and enables efficient inference. Experiments show that NaP attains higher success rates and faster inference while preserving natural motion across diverse tasks.

Summary

Key Idea

NaP-Control is a latent noise optimization framework that combines reinforcement learning (RL) with a task-agnostic diffusion-based motion prior to achieve precise, whole-body character control. Instead of relying on slow, iterative gradient guidance at test-time, it learns to navigate the diffusion prior’s input noise space to generate task-driven, physically plausible behaviors.

Key Challenges

Current diffusion-based policies generate expressive motions but often rely on gradient-based test-time guidance to satisfy task objectives, which is computationally expensive and can be fragile in complex environments. Furthermore, most diffusion models are trained purely offline, limiting their ability to adapt to unseen, closed-loop physical interactions like uneven terrain traversal or contact-rich object manipulation.

How: Navigating the Latent Noise Manifold of Natural Human Motion Prior

Diffusion Prior as Action Manifold: The method uses a pretrained, task-agnostic diffusion model as a high-fidelity motion manifold that encapsulates natural human-like movements.
RL-Based Noise Navigation: Inspired by DSRL, a compact RL policy (trained via PPO) learns to predict the initial latent noise for the diffusion process. This directs the generative process toward actions that maximize task-specific rewards while remaining within the natural motion distribution.
Environment Interaction: Unlike offline methods, NaP-Control interacts directly with a physics simulator during training, allowing it to correct motions based on environmental feedback and adapt to scenarios not seen in the original motion data.

Highlight: NaP-Control eliminates expensive test-time iterative guidance, achieving up to 7.7x faster inference and higher success rates than the existing diffusion-based policy while maintaining superior motion naturalness.

Framework

Photo of two running dogs, lossily compressed using the DiffC algorithm — Framework overview. (a) The RL policy $\pi_{\theta}$ receives environment and proprioceptive states from the physics simulator. (b) The actor learns to predict optimal noise $\boldsymbol{\omega} \in \mathcal{W}$ aligned with task goals. (c) These predicted noises are denoised and decoded into executable actions $\mathbf{a}$ via a pretrained diffusion prior and a latent action decoder. Resulting transitions are then used to iteratively optimize the noise navigation policy with RL. (d) Acting as a natural motion manifold, the diffusion prior can map predicted noises to future states $\mathbf{s}^{\text{prop}}$ and latent actions $\mathbf{z}$ conditioned on the trajectory history $\mathcal{H}$ .

Results

We showcase the effectiveness of NaP-Control in (a) far goal reaching, (b) agile right hand reaching, (c) velocity control, (d) object interaction tasks, as well as its adaptation on uneven terrains. Notably, NaP-Control successfully adapts to complex terrains despite the underlying motion prior being trained exclusively on flat ground.

Far Goal Reaching

Agile Right Hand Reaching

Velocity Control

Object Interaction

BibTeX citation

@misc{chen2026napcontrolnavigatingdiffusionprior,
      title={NaP-Control: Navigating Diffusion Prior for Versatile and Fast Character Control},
      author={Chia-Wen Chen and Yan Wu and Korrawe Karunratanakul and Siyu Tang},
      year={2026},
      eprint={2605.20209},
      archivePrefix={arXiv},
      primaryClass={cs.GR},
      url={https://arxiv.org/abs/2605.20209},
}