Skip to main content
Move the World.
language generator

Lead Image © grumpybox / Adobe Stock

OpenAI's new machine learning language generator, GPT-3, is currently an internet darling. With a catholic knowledge of the English canon, drawn from the vast corners of the internet, the AI can generate a piece of writing that can, at times, read as fine as any a human could compose.

It's a comprehensive step beyond AI's usual playing with language, and perhaps a hint that machine learning could soon assume powerful roles in writing, much like those it is assuming in medicine and robotics.

GPT-3 is the most powerful language generator yet made, according to MIT's Technology Review. Inside any neural network's black box are parameters, the guardrails for its training: GPT-2, released last year, had a massive 1.5 billion parameters; GPT-3, in comparison, has 175 billion — one of those hard-to-comprehend numbers.

Sucking this dataset, Charybdis-like, into its maw, GPT-3 can now spit out, based on all it has "read," whatever words should follow a given prompt.

Styles and Shortcomings

GPT-3 uses its vast dataset to apply a mathematical prediction as to what words, and in what order, will best complete a given prompt. Some of the results have been deeply impressive, despite that ghostly quality AI writing seems to always contain.

If given a prompt with a sci-fi bent — say, the opening line to George Orwell's 1984, from the Guardian's Alex Hern — it will return a suitably sci-fi result. 

"It was a bright cold day in April, and the clocks were striking thirteen," Orwell begins.

"I was in my car on my way to a new job in Seattle. I put the gas in, put the key in, and then I let it run," the language generator continued. "I just imagined what the day would be like. A hundred years from now. In 2045, I was a teacher in some school in a poor part of rural China. I started with Chinese history and history of science."

It's a hair disjointed, but read altogether, it does feel like a particularly promising freshman comp assignment. 

People playing with GPT-3 have prompted poetry and prose in the styles of certain authors, composed music, and created code. It's even written an article on itself. (Your correspondent will fight an AI to keep his job.)

Little wonder that the Twitter cage is rattling.

GPT-3 is not a perfect mimic, or an artificial general intelligence, however. Kevin Lacker's Turing test of GPT-3 reveals some of its shortcomings.

The language generator has little issue predicting answers to trivia-style questions; it knows who won the 1995 World Series, how many eyes a giraffe has, and the human life expectancy in the United States.

 It proved pretty good at answering common sense questions, too, knowing that an elephant is heavier than a mouse, and a pop can heavier than a paperclip. But give it a prompt that's a bit odd — Lacker asked, is a pencil heavier than a toaster? — and you can trip it up. 

(GPT-3's response: the pencil is heavier. This is likely because the literature on comparing pencils and toasters is probably pretty ... light.)

GPT-3 uses its vast dataset to apply a mathematical prediction as to what words will best complete a given prompt.

Interestingly, it won't admit it is wrong, and it won't call out your question if it's nonsensical. When asked how to sporgle a morgle, it replied "with a sporgle" (uh, duh).

But even OpenAI's founder, Sam Altman, tried to temper things a bit. 

"The GPT-3 hype is way too much," he tweeted. "It's impressive (thanks for the nice compliments!) but it still has serious weaknesses and sometimes makes very silly mistakes. AI is going to change the world, but GPT-3 is just a very early glimpse. We have a lot still to figure out."

Still, while it's true that GPT-3 is not an artificial general intelligence — or even intelligent, in, like, a philosophical sense ("Music is the most advanced form of mathematics" is a sophomoric, one-blunt insight, GPT-3) — that does not mean it's as simple as a chatbot toy.

With the ability to write, with varying levels of convincingness, everything from fiction to music, poetry to technical information, journalism to code, GPT-3 could prove to be a powerful tool. Any task that requires the written word could be augmented — or even automated — with a high quality language generator.

And GPT-3 might just be the lede.

Up Next

Uprising
Can Humans Figure Out How Deep Learning AI Thinks?
deep learning ai
Uprising
Can Humans Figure Out How Deep Learning AI Thinks?
Deep learning AI is becoming more complex, capable, and impenetrable, but these scientists are attempting to break the black box.

Deep learning AI is becoming more complex, capable, and impenetrable, but these scientists are attempting to break the black box.

Challengers
Can AI Solve Our Biggest Problems?
Can AI Solve Our Biggest Problems?
Watch Now
Challengers
Can AI Solve Our Biggest Problems?
Vicarious believes smart machines could solve virtually every problem humans can’t.
Watch Now

Scott Phoenix, cofounder of Vicarious, believes smart machines could one day cure cancer, create new forms of energy, and solve virtually every problem that humans simply can’t. With the support of investors like Jeff Bezos and Mark Zuckerberg, Vicarious is striving to bring about this future in our lifetimes.

AI & Ethics
"Moral Machine" Picks Up On Human Ethics
Moral Machine
AI & Ethics
"Moral Machine" Picks Up On Human Ethics
By feeding an AI a diet of books, news articles, and religious texts, researchers were able to create a “moral machine” reflecting the ethics of a society.

By feeding an AI a diet of books, news articles, and religious texts, researchers were able to create a “moral machine” reflecting the ethics of a society.

Dispatches
AI Will Make You Smarter
AI Will Make You Smarter
Dispatches
AI Will Make You Smarter
Artificial intelligence will multiply your own intelligence, in ways that will surprise you.
By Terrence Sejnowski

Artificial intelligence will multiply your own intelligence, in ways that will surprise you.

Robotics
Firefighting Robot Helps Battle Blaze in Los Angeles
Firefighting Robot
Robotics
Firefighting Robot Helps Battle Blaze in Los Angeles
The U.S.’s first firefighting robot helped keep LAFD firefighters safe while they battled a blaze in downtown Los Angeles.

The U.S.’s first firefighting robot helped keep LAFD firefighters safe while they battled a blaze in downtown Los Angeles.

Clean Energy
New Nuclear Fuel Could Replace Uranium in Reactors
nuclear fuel
Clean Energy
New Nuclear Fuel Could Replace Uranium in Reactors
Nuclear power could become more accessible thanks to a new nuclear fuel called ANEEL that combines thorium with uranium.

Nuclear power could become more accessible thanks to a new nuclear fuel called ANEEL that combines thorium with uranium.

Medical Innovation
This Adjustable Heart Valve Would Grow as a Child Ages
heart valve
Medical Innovation
This Adjustable Heart Valve Would Grow as a Child Ages
A new, prototype artificial heart valve can adjust to a child’s growing body, potentially sparing them from multiple open-heart surgeries before adulthood.

A new, prototype artificial heart valve can adjust to a child’s growing body, potentially sparing them from multiple open-heart surgeries before adulthood.

Dispatches
Training the Body to Fight Off Drug-Resistant Bacteria
Training the Body to Fight Off Drug-Resistant Bacteria
Dispatches
Training the Body to Fight Off Drug-Resistant Bacteria
A new strategy, called host-targeted defense, could help solve antibiotic resistance by upgrading the immune system.
By Zahidul Alam

A new strategy, called host-targeted defense, could help solve antibiotic resistance by upgrading the immune system.

Coded
Hacking the Future
Hacking the Future
Watch Now
Coded
Hacking the Future
How do we make sure the next generation of hackers uses their talents for good?
Watch Now

In our hyper-connected world, hacking is a superpower. And Nico Sell wants to make sure that power ends up in the right hands. She started Rootz Asylum to teach kids how to hack and encourage them to use their new-found talents for good.