You win this street game by getting “hit” by a self-driving car

A game that challenges pedestrians to avoid being detected by an AI could ensure tomorrow’s self-driving cars “see” all the people around them, regardless of what they look like or what they’re doing. 

The challenge: Self-driving cars could make our streets safer by eliminating human error from the transportation equation, but training the AIs that control them is a huge undertaking.

“This sort of data is so hard to catch and predict.”

Daniel Coppen

Not only do these systems need to understand traffic laws, they also need to be able to adapt to variables such as construction, emergency vehicles, inclement weather, and pedestrians of all shapes and sizes.

“There could be children on the street who are dressed up. It could be someone carrying a box. It could be people in wheelchairs,” said Daniel Coppen, creative director of design / art unit Playfool. “All these different use cases … they’re so vast, and they’re so unpredictable, and this sort of data is so hard to catch and predict.”

The idea: Coppen recently teamed up with media artist Tomo Kihara to develop “How (not) to get hit by a self-driving car,” a street-based game designed to get people thinking about the existing limitations of self-driving cars — and potentially help overcome them. 

“We really hope that our experience will … allow these self-driving car systems to become smarter and safer,” said Coppen.

How it works: The game challenges passersby to make it from one end of a play area to the other without being identified as a pedestrian by a camera linked to an object-detection algorithm.

A large screen displays what the AI “sees” while players try to fool it with cartwheels, disguises, and other evasive maneuvers.

Players who beat the AI can choose to have their anonymized footage deleted or retained by the game’s makers, who are now looking for research institutions to share the dataset with in the hope it will be used to train self-driving cars.

Note: Just because you can fool the AI in the game doesn’t mean one of today’s self-driving cars would actually hit you if you crab-walked toward it.

The researchers don’t know for sure that any self-driving car developers are using the same object-detection algorithm — the Single Shot Detector (SSD) — that’s in the game to train their vehicles. However, they also aren’t ruling it out.

“[E]xperts suggest that [developers] train their systems based on algorithms like SSD or YOLO, a similar detection algorithm,” they write. “These are two of the most known algorithms in object detection, both of which have undergone extensive testing and benchmarking.”

Looking ahead: “How (not) to get hit by a self-driving car” debuted on the streets of Bristol in July. The researchers are now preparing to follow up that five-day installation with a deployment at Science Gallery London on November 17.

Their ultimate goal is to do a “world tour” with the game so that they can collect the widest variety of data possible. 

“We want to do the same game, but on the bustling streets of Mumbai or really crowded streets in Tokyo, see what kind of data it’ll generate, what kind of play that will result in,” said Kihara.

We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].