How to spot a deepfake? It’s all in the eyes.

Researchers have created a tool capable of spotting deepfakes with 94% accuracy — given the right conditions.
Subscribe to Freethink on Substack for free
Get our favorite new stories right to your inbox every week

Spotting deepfakes is no easy task — just ask Tom Cruise. (There’s a Mission: Impossible joke in here somewhere…)

Designed with sophisticated deep-learning algorithms, computer scientists expect deepfakes to keep getting more convincing, making spotting deepfakes ever more difficult — and valuable.

“We all assume that there will be a point where there’s no way to tell the difference. I mean, for visual effects, I think you can get pretty close already. It’s just the question of how much effort you put into it,” USC associate professor and Pinscreen founder Hao Li told PBS NewsHour.

“But in terms of content that (…) can be created by anyone, I think it’s getting very close to the point.”

Deepfakes aren’t all bad; they can be abused to produce porn and political chaos, but a deepfake tool can bring the past to life or star in TV comedies. But regardless of intent, spotting deepfakes is still essential if we don’t want to lose reality. 

Fortunately, researchers at The University at Buffalo have developed a tool capable of spotting deepfakes with 94% accuracy, the university announced in a release.

The Secret’s in the Eyes

Spotting deepfakes is a nuance game. 

“The cornea is almost like a perfect semisphere and is very reflective,” UB computer science professor Siwei Lyu said in a statement. Because of that reflectivity, the eyes in a photograph will reflect back anything emitting light.

“The two eyes should have very similar reflective patterns because they’re seeing the same thing. It’s something that we typically don’t typically notice when we look at a face,” Lyu said.

Deepfakes, for some reason, often fail to nail these reflections — perhaps because they use composite images to craft their creations. Lyu and his team’s tool takes advantage of this weakness.

The tool — accepted for presentation at the IEEE International Conference on Acoustics, Speech and Signal Processing this June in Toronto, and published as a pre-print — begins by mapping the face. Then it turns its attention to the eyes, then the actual eyeballs themselves, before drilling down further and focusing, pixel-by-pixel, on the light reflected in the corneas.

If there’s oddities or deviations in shape, intensity, or other features of the reflections, you’re (likely) spotting deepfakes.

To train their model, the researchers used a database of deepfaked faces from www.thispersondoesnotexist.com, as well as images of real people from Flickr Faces-HQ. 

Although, all of the images were what the researchers describe as in a “portrait setting” — well-lit, facing the camera, and at 1,024 x 1,024 pixels.

Lyu’s model has some other limitations, too. Since you’re examining the light reflected in the eyes you need a source of light to be reflected. And you need two eyes, at that — if one is missing or obscured, the tool has nothing to compare.

It also doesn’t examine the shape of the eyes, or what is being reflected in them, per UB; it’s only looking at the pixels, and those pixels could be made to match each other by a sufficiently dedicated and detail-oriented deepfaker.

The Eyes Have It

The need for spotting deepfakes runs from the intensely intimate to the geopolitical. 

“Unfortunately, a big chunk of these kinds of fake videos were created for pornographic purposes, and that (caused) a lot of … psychological damage to the victims,” Lyu said. “There’s also the potential political impact, the fake video showing politicians saying something or doing something that they’re not supposed to do. That’s bad.”

And the deepfakes are proliferating. According to a report from CB Insights, the number of deepfakes identified online jumped from around 24,000 to over 49,000 — from January to June of 2020.

As they do, so will the ranks of researchers looking to spot them. 

We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].

Subscribe to Freethink on Substack for free
Get our favorite new stories right to your inbox every week
Related
The missing tech case for how we create an era of abundance
AI and other new technologies could make things that are costly and scarce today, cheap and abundant for all tomorrow.
Why America reinvents itself every 80 years — and is doing so again
Three separate theories help explain why America enters a period of great progress every 80 years — and why another is coming soon.
How DeepSeek rewrote the rules of the AI race
Chinese startup DeepSeek has proven that vast quantities of capital and cutting-edge chips aren’t prerequisites for world-class AI.
Kevin Kelly points a new way forward into the Age of AI
One of the most original and optimistic thinkers in America helps build out some big through lines on what’s possible with AI in the next 25 years.
The artifact isn’t the art: Rethinking creativity in the age of AI
ChatGPT’s Studio Ghibli imitations invite questions about the creative value of people and what we really mean when we talk about creativity.
Up Next
americas cup ai
Subscribe to Freethink for more great stories