Breaking

Humanoid Locomotion Learned Directly From Raw Depth Pixels, Accepted to RSS 2026

June 29, 2026 at 10:15 EDT

Robotics
Research & Papers

A research team in China has released "Now You See That," an end-to-end method that learns humanoid locomotion control directly from the raw depth pixels of stereo cameras. Using a single policy, it can traverse diverse and difficult terrain such as stairs, rubble and gaps, and the work has been accepted to RSS 2026, a leading robotics conference.

RSS 2026 · Harbin Institute of Technology × HONOR Robotics

"Now You See That": Humanoids That Walk by Sight Alone

An end-to-end policy drives whole-body humanoid locomotion straight from raw stereo depth images — clearing high platforms, wide gaps, long staircases and rubble with a single unified controller.

unified policy for all terrains — no per-terrain controllers

humanoid types validated, each with a different stereo camera

noise sources modeled: stereo holes, depth, Perlin, Gaussian

Two approaches to seeing the ground

Conventional: LiDAR elevation map

Accumulating odometry drift
Low update rate & latency
Map diverges from real footing

This method: raw depth vision

End-to-end stereo depth input
Richer scene information
Sim-to-real gap closed by augmentation

Vision-aware behavior distillation

A teacher trained on a perfect height map hands its behavior to a student that sees only noisy raw depth.

TEACHER

Privileged RL on ideal height map

→

DISTILL

Multi-critic / multi-discriminator + per-terrain rewards

→

STUDENT

One policy on noisy raw depth

Edge over LiDAR / height-map baselines

Five capabilities reported as advantages over prior work

End-to-end vision input Comprehensive noise modeling Long-term deployment Fine locomotion Extreme parkour

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.