MIT’s new AI can make holograms in real-time

It’s efficient enough to run on smartphones — no supercomputers necessary.

Holograms could make virtual reality more immersive, improve our 3D printers, and even help doctors diagnose and treat patients — if only they weren’t so difficult to create.

“It’s often been said that commercially available holographic displays will be around in 10 years, yet this statement has been around for decades,” MIT researcher Liang Shi told MIT News.

Now, Shi and his colleagues at MIT have developed a technique to generate holograms in real-time — and it’s so efficient, it could be done on a laptop or smartphone.

What Is a Hologram?

Pop culture has made the term “hologram” synonymous with deceased rapper Tupac Shakur’s “performance” at Coachella 2012, but that wasn’t technically a hologram (it was a mirror-based optical illusion called “Pepper’s Ghost”).

A hologram is a flat image that appears to be three-dimensional.

Consider a standard photograph of an apple. No matter how you move your head or angle the photograph, it’s always going to look like the same flat image of an apple.

Flat pictures like this are created by recording the light waves reflecting off whatever is in a camera’s view when its shutter was clicked.

But a hologram of an apple would have depth. As you moved your head or the hologram itself, you’d feel like you were seeing new angles of the fruit.

Holograms use both the brightness and the phase of each light wave to give the viewer the sense that they’re looking at something three-dimensional — even though they aren’t.

A Hologram-Making AI

Traditionally, holograms were created using laser beams, but the images resulting from that technique could only be displayed as hard copies that were difficult to reproduce. The method couldn’t translate to video, either — only static images.

Computer-based techniques for making holograms can overcome those limitations, but they involve running physics-based simulations, which require a lot of processing power.

We are amazed at how well it performs.

Wojciech Matusik

It could take a supercomputer cluster minutes to generate a single hologram, and the final result still might not be photorealistic.

To speed up the process, while cutting down on the computational burden, MIT has developed a new AI-based technique they call “tensor holography.”

The researchers started by creating a training dataset of 4,000 computer-generated images and their matching photorealistic holograms. Each of the images included the color and depth information for every pixel.

They then trained an AI with this data, teaching it how to generate a hologram for almost any given 2D image. By the time it was finished, it could produce a photorealistic hologram in milliseconds.

“We are amazed at how well it performs,” researcher Wojciech Matusik told MIT News.

The AI requires less than 1 MB of memory — a fraction of what’s on most smartphones. The tech needed to calculate depth information also comes standard on many of today’s phones, meaning the devices could easily support the hologram-making process.

“It’s a considerable leap that could completely change people’s attitudes toward holography,” Matusik said. “We feel like neural networks were born for this task.”

We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at

A celebrated AI has learned a new trick: How to do chemistry
We had a challenging question for the AI AlphaFold – had its structural training set taught it some chemistry?
boosting crop yield
How artificial intelligence is boosting crop yield to feed the world
The Gene Ranking Artificial Intelligence Network (GRAIN) identifies genes that act at a fundamental level in crop metabolism.
How to be a techno-optimist
Technology will not save the world, and it is inherently neither good nor bad. But, when tech is coupled to human virtue, good will prevail.
underwater garden
Dive into the world’s first underwater garden
An underwater garden off the coast of Italy is introducing the world to a new type of sustainable agriculture.
Google has not created sentient AI — yet
Google can make AI seem sentient, but the AI has no understanding of what it is saying. But humans are easily fooled by conversational AI.
Up Next
underwater glider
Subscribe to Freethink for more great stories