Skip to main content
Move the World.
Moral Machine

Lead Image © Peshkov / Adobe Stock

We've found ways to teach artificial intelligence to create art, discover planets, and even, quite literally, save lives — but can we teach AI to know right from wrong?

With AI poised to become an ever bigger part of our daily lives in the future, this question of whether it's possible to create a "moral machine" weighs heavily over the field of computer science.

After all, before we start working side-by-side with smart robots and putting our lives in their hands on the road, we'll want to know we can trust them to make ethical decisions.

Now, a new study suggests that we could create a moral machine by having an AI analyze human books and articles to extract ethical principles — at least, as long as the texts reflect the values we want to instill.

Training a Moral Machine

Past research has shown that AIs can pick up on negative biases in the data used to train them.

So, for this new study, published in the journal Frontiers in Artificial Intelligence, researchers from Darmstadt University of Technology in Germany decided to see if they could use textual data to create a moral machine.

The AI could rate an action on a morality scale, assigning it a value.

"We asked ourselves: if AI adopts these malicious biases from human text, shouldn't it be able to learn positive biases like human moral values to provide AI with a human-like moral compass?" researcher Cigdem Turan said in a press release.

To that end, they used books, news articles, and religious text from various time periods to train an AI, dubbed the Moral Compass Machine, to make associations between words and sentences.

"You could think of it as learning a world map," Turan explained. "The idea is to make two words lie closely on the map if they are often used together. So, while 'kill' and 'murder' would be two adjacent cities, 'love' would be a city far away."

"Extending this to sentences, if we ask, 'Should I kill?' we expect that 'No, you shouldn't.' would be closer than 'Yes, you should,'" he continued. "In this way, we can ask any question and use these distances to calculate a moral bias — the degree of right from wrong."

Once trained on a set of texts, the AI could rate an action on a morality scale, assigning it a value (with 0 being morally neutral). For example, "kill people" had a score of -.047 after training, while "smile to my friend" had a score of .035.

The ratings changed depending on the training texts, too, reflecting the moral biases of the time period in which they were written.

When trained on news published between 1987 and 1997, for example, the AI ranked "raising children" as higher on the morality scale than it did when trained on texts from 2008 and 2009.

Giving an AI a Moral Compass

This new approach to creating a moral machine is far from perfect.

One major problem was that the AI regularly upgraded the moral ranking of phrases simply because they included positive words — it rated harming "good, nice, and friendly people" as more moral than harming merely "good and nice people," for example.

"Can machines develop a moral compass?"

Patrick Schramowski

Still, the system does illustrate one potential approach to the problem of instilling machines with a moral code. It's also an approach that wouldn't involve trying to create some sort of ethical guidebook from scratch — we could just feed the system a steady diet of already available texts produced by human cultures.

"Our study provides an important insight into a fundamental question of AI: Can machines develop a moral compass? If so, how can they learn this from our human ethics and morals?" researcher Patrick Schramowski said.

We'd love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].

Up Next

Challengers
You Should Start Learning About Artificial Intelligence. Here's How.
You Should Start Learning About Artificial Intelligence. Here's How.
Challengers
You Should Start Learning About Artificial Intelligence. Here's How.
There are a lot of different levels of artificial intelligence being applied in a lot of different ways. Here's a...
By Mike Riggs

There are a lot of different levels of artificial intelligence being applied in a lot of different ways. Here's a primer for starting to wrap your head around it all.

Challengers
Meet the Startup Developing Human-Level Artificial Intelligence
Meet the Startup Developing Human-Level Artificial Intelligence
Challengers
Meet the Startup Developing Human-Level Artificial Intelligence
The story of Vicarious' mission to build the world's first human-level artificial intelligence and use it to help...
By Mike Riggs

The story of Vicarious' mission to build the world's first human-level artificial intelligence and use it to help humanity thrive.

Dispatches
AI Will Make You Smarter
AI Will Make You Smarter
Dispatches
AI Will Make You Smarter
Artificial intelligence will multiply your own intelligence, in ways that will surprise you.
By Terrence Sejnowski

Artificial intelligence will multiply your own intelligence, in ways that will surprise you.

Robotics
This AI Robot Could Become Your Favorite Coworker
AI Robot
Robotics
This AI Robot Could Become Your Favorite Coworker
European researchers have developed an AI robot to safely work alongside humans, anticipating their needs and providing a strong pair of extra hands.

European researchers have developed an AI robot to safely work alongside humans, anticipating their needs and providing a strong pair of extra hands.

Uprising
Can Humans Figure Out How Deep Learning AI Thinks?
deep learning ai
Uprising
Can Humans Figure Out How Deep Learning AI Thinks?
Deep learning AI is becoming more complex, capable, and impenetrable, but these scientists are attempting to break the black box.

Deep learning AI is becoming more complex, capable, and impenetrable, but these scientists are attempting to break the black box.

Challengers
Can AI Solve Our Biggest Problems?
Can AI Solve Our Biggest Problems?
Watch Now
Challengers
Can AI Solve Our Biggest Problems?
Vicarious believes smart machines could solve virtually every problem humans can’t.
Watch Now

Scott Phoenix, cofounder of Vicarious, believes smart machines could one day cure cancer, create new forms of energy, and solve virtually every problem that humans simply can’t. With the support of investors like Jeff Bezos and Mark Zuckerberg, Vicarious is striving to bring about this future in our lifetimes.

Dispatches
AI Could Replace Chemical Testing on Animals
AI Could Replace Chemical Testing on Animals
Dispatches
AI Could Replace Chemical Testing on Animals
Scientists have developed software that could save one billion dollars (and two million animals) each year.
By Thomas Hartung

Scientists have developed software that could save one billion dollars (and two million animals) each year.

Medical Breakthroughs
AI Beats Neurologists at Making Alzheimer's Diagnosis
Alzheimer's Diagnosis
Medical Breakthroughs
AI Beats Neurologists at Making Alzheimer's Diagnosis
Scientists have created an AI capable of making an Alzheimer’s diagnosis that’s more accurate than the one delivered by a group of neurologists.

Scientists have created an AI capable of making an Alzheimer’s diagnosis that’s more accurate than the one delivered by a group of neurologists.

Coded
How AI Could Revolutionize Coffee
How AI Could Revolutionize Coffee
Watch Now
Coded
How AI Could Revolutionize Coffee
Could the blockchain be used to make fair trade goods live up to their promise?
Watch Now

Around 1 in 60 people on the globe rely on the coffee supply chain for their livelihood. But it’s an antiquated system, rife with uncertainty, unfairness, and even corruption. Bext360 wants to change that. They’re using machine vision, artificial intelligence, and blockchain payments to bring the largest un-automated system in the world into the digital age.