Modeling and Understanding Neural Weights of Humanoid Policies with Diffusion
Researchers generate deployable Unitree G1 humanoid controllers by diffusing learned policy weights.
The paper studies whether high-performing humanoid control networks can be generated by modeling their weights rather than training each policy conventionally. A linear autoencoder compresses actor parameters and a Diffusion Transformer models that latent distribution, using networks collected from reinforcement learning on the Unitree G1. Generated policies were deployed for real-world walking, dancing, and spin kick. The authors also report a structural pattern in first-layer weights that matches the robot's kinematic prior under both reinforcement learning and imitation learning.
- A latent diffusion model generates humanoid control networks directly in weight space.
- A linear autoencoder and Diffusion Transformer were trained on Unitree G1 RL policies.
- Generated policies were deployed for walking, dancing, and spin kick.
- First-layer weights reflect a kinematic prior under both RL and imitation learning.
Full article246 words · extracted from arxiv.org · click to collapse
Recent advances in robot learning have driven rapid progress in humanoid control, enabling increasingly agile behaviors in the real world. However, humanoid control networks are typically obtained through conventional training procedures, while comparatively little attention has been given to directly modeling the distribution of their learned weights and generating functional networks in weight space. In this work, we investigate whether high-performing humanoid control networks can be generatively modeled directly through their weights. We adopt a latent diffusion framework that couples a linear autoencoder with a Diffusion Transformer. The autoencoder compresses actor network parameters into a compact latent space, and the Diffusion Transformer models the distribution of these latents. Trained on networks collected from reinforcement learning for the Unitree G1 robot, our approach generates deployable networks that perform various motions in the real world, including walking, dancing, and spin kick. Beyond generation, we further study the learned weight space to better understand the structure captured by these humanoid control networks. We identify a structural principle in the first-layer weights that consistently reflects the robot's kinematic prior. Notably, this kinematic prior emerges under both Reinforcement Learning (RL) and Imitation Learning (IL) training, suggesting that it is not specific to a particular optimization process but may instead reflect common knowledge associated with humanoid embodiment. Together, these results demonstrate the feasibility of generative modeling in humanoid network weight space, while providing initial insights into the information encoded by their learned weights. Additional results are available on our project website: https://humanoid-weight-diffusion.github.io/
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2610.20606