These robot legs taught themselves to walk

A new twist on reinforcement learning could make it easier to train tomorrow’s robots.
Subscribe to Freethink on Substack for free
Get our favorite new stories right to your inbox every week

“Cassie,” a bot made by Agility Robotics, is essentially a pair of robot legs.

But Cassie has taught itself to walk — thanks to UC Berkeley researchers’ unique twist on reinforcement learning.

Why it matters: Legged robots are better at navigating tough terrain than their wheeled counterparts.

That gives them countless applications — from search and rescue to off-world exploration — and this new technique could make it easier to train the robots for any of those tasks.

Treat for trick: Reinforcement learning is a commonly used technique for training AI robots to walk.

Rather than giving an AI control over a robot right away — and risking it leading the expensive equipment right down a set of stairs — researchers will create a virtual environment designed to mimic the physics of the real one.

Their AI version will then learn to walk in that environment through a process of trial and error. It receives a reward for desired actions, and a penalty when it does something wrong.

From that feedback, the AI eventually masters walking in the simulation — and then it’s given control of an actual robot.

The challenge: It’s impossible to perfectly mimic the real world in a simulation, and even tiny differences between the virtual world and the real one can affect the robot’s performance.

That means researchers must often manually adjust their AI once it’s already reached the robot stage — which can be a time-consuming process.

Doubling up: Rather than letting the AI powering their robot legs learn to walk in one simulation, the Berkeley team used two virtual environments.

In the first, the AI learned to walk by trying out different actions from a large, pre-programmed library of robot movements. During this training, the dynamics of the simulated environment would change randomly — sometimes the AI would experience less ground friction or find itself tasked with carrying a load.

The robot legs could walk across slippery terrain, carry loads, and even recover when shoved.

This technique, called “domain randomization,” was incorporated into the training to help the AI think on its feet once it encountered the sometimes-unpredictable real world.

In the second environment, the AI tested out what it learned in a simulation that very closely mimicked the physics of the real world.

The accuracy of this simulation was only possible by sacrificing processing speed — it would’ve taken too long for a robot to learn how to walk in it, but it did serve as a useful testing ground before making the leap to the real world.

After that, the AI was given control over the robot legs and had very little trouble using them. It could walk across slippery terrain, carry loads, and even recover when shoved — all without any extra adjustments from the researchers.

First steps: The robot legs will need more training before they can have any real use outside the research lab. The Berkeley team now plans to see if they can replicate the bot’s smooth sim-to-real transfer with more dynamic and agile behaviors.

We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].

Subscribe to Freethink on Substack for free
Get our favorite new stories right to your inbox every week
Related
A personal assistant for everyone: The promise of ambient AI
We’re leaving the app era and entering the age of ambient AI: intelligent help that’s always on, but never in the way.
Gen Z: We must resist the temptation to cheat on everything
Adopting the “cheat on everything” mentality — treating thinking as a burden AI can eliminate — is not only wrong, it’s dangerous.
There are no new ideas in AI — only new datasets
Our next AI breakthrough will come when we unlock a source of data we’ve either overlooked or never fully harnessed.
Albert Einstein said automation caused the Great Depression. It didn’t.
Einstein blamed automation for the widespread unemployment of the Great Depression, but his reasoning was based on a false premise.
Has AI made “learn to code” obsolete?
Freethink talks to the creator of the world’s most popular AI coding assistant to find out whether learning to code is still worthwhile.
Up Next
lost art
Subscribe to Freethink for more great stories