Forget Boston Dynamics. This robot taught itself to walk
Virtual restrictions: Reinforcement learning has been used to train bots to run in simulations, but it’s difficult to apply this skill to the real world. “A lot of the videos you see of virtual agents aren’t realistic at all,” says Chelsea Finn, an AI and robotics researcher at Stanford University who was not involved in the work. Small differences between the simulated laws of physics in a virtual environment and the real laws of physics outside the environment – for example, how the friction between a robot’s feet and the floor works – can lead to big mistakes when a robot tries to apply what it has learned. A heavy two-legged robot can lose its balance and fall if its movements are just a tiny bit different.
Double simulation: However, it would be dangerous to train a large robot through trial and error in the real world. To work around these issues, the Berkeley team used two layers of the virtual environment. In the first version, a simulated version of Cassie learned to walk using a large existing database of robot movements. This simulation was then transferred to a second virtual environment called SimMechanics, which mirrors the real physics with a high degree of accuracy – but at a price for running speed. It was only when Cassie appeared to be walking well that the learned walking model was loaded into the actual robot.
The real Cassie was able to run with the model learned in the simulation without additional fine-tuning. It could walk over rough and slippery terrain, carry unexpected loads, and recover from a thrust. During the test, Cassie also damaged two motors in the right leg, but was able to adjust his movements to compensate for this. Finn thinks it’s exciting work. Edward Johns, who heads the Robot Learning Lab at Imperial College London, agrees. “This is one of the most successful examples I’ve seen,” he says.
The Berkeley team hopes this approach will expand Cassie’s movement repertoire. But don’t expect a dance-off anytime soon.