These robot legs taught themselves to walk

A new twist on reinforcement learning could make it easier to train tomorrow’s robots.
Sign up for the Freethink Weekly newsletter!
A collection of our favorite stories straight to your inbox

“Cassie,” a bot made by Agility Robotics, is essentially a pair of robot legs.

But Cassie has taught itself to walk — thanks to UC Berkeley researchers’ unique twist on reinforcement learning.

Why it matters: Legged robots are better at navigating tough terrain than their wheeled counterparts.

That gives them countless applications — from search and rescue to off-world exploration — and this new technique could make it easier to train the robots for any of those tasks.

Treat for trick: Reinforcement learning is a commonly used technique for training AI robots to walk.

Rather than giving an AI control over a robot right away — and risking it leading the expensive equipment right down a set of stairs — researchers will create a virtual environment designed to mimic the physics of the real one.

Their AI version will then learn to walk in that environment through a process of trial and error. It receives a reward for desired actions, and a penalty when it does something wrong.

From that feedback, the AI eventually masters walking in the simulation — and then it’s given control of an actual robot.

The challenge: It’s impossible to perfectly mimic the real world in a simulation, and even tiny differences between the virtual world and the real one can affect the robot’s performance.

That means researchers must often manually adjust their AI once it’s already reached the robot stage — which can be a time-consuming process.

Doubling up: Rather than letting the AI powering their robot legs learn to walk in one simulation, the Berkeley team used two virtual environments.

In the first, the AI learned to walk by trying out different actions from a large, pre-programmed library of robot movements. During this training, the dynamics of the simulated environment would change randomly — sometimes the AI would experience less ground friction or find itself tasked with carrying a load.

The robot legs could walk across slippery terrain, carry loads, and even recover when shoved.

This technique, called “domain randomization,” was incorporated into the training to help the AI think on its feet once it encountered the sometimes-unpredictable real world.

In the second environment, the AI tested out what it learned in a simulation that very closely mimicked the physics of the real world.

The accuracy of this simulation was only possible by sacrificing processing speed — it would’ve taken too long for a robot to learn how to walk in it, but it did serve as a useful testing ground before making the leap to the real world.

After that, the AI was given control over the robot legs and had very little trouble using them. It could walk across slippery terrain, carry loads, and even recover when shoved — all without any extra adjustments from the researchers.

First steps: The robot legs will need more training before they can have any real use outside the research lab. The Berkeley team now plans to see if they can replicate the bot’s smooth sim-to-real transfer with more dynamic and agile behaviors.

We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].

Sign up for the Freethink Weekly newsletter!
A collection of our favorite stories straight to your inbox
Related
AI chatbots may ease the world’s loneliness (if they don’t make it worse)
AI chatbots may have certain advantages when roleplaying as our friends. They may also come with downsides that make our loneliness worse.
Will AI supercharge hacking — if it hasn’t already?
The future of hacking is coming at us fast, and it isn’t clear yet whether AI will help attackers and defenders more.
No, LLMs still can’t reason like humans. This simple test reveals why.
Most AI models are incredible at taking tests but easily bamboozled by basic reasoning. “Simple Bench” shows us why.
The future of fertility, from artificial wombs to AI-assisted IVF
A look back at the history of infertility treatments and ahead to the tech that could change everything we thought we knew about reproduction.
“Model collapse” threatens to kill progress on generative AIs
Generative AIs start churning out nonsense when trained on synthetic data — a problem that could put a ceiling on their ability to improve.
Up Next
lost art
Subscribe to Freethink for more great stories