MIT’s new AI can make holograms in real-time

Holograms could make virtual reality more immersive, improve our 3D printers, and even help doctors diagnose and treat patients — if only they weren’t so difficult to create.

“It’s often been said that commercially available holographic displays will be around in 10 years, yet this statement has been around for decades,” MIT researcher Liang Shi told MIT News.

Now, Shi and his colleagues at MIT have developed a technique to generate holograms in real-time — and it’s so efficient, it could be done on a laptop or smartphone.

What Is a Hologram?

Pop culture has made the term “hologram” synonymous with deceased rapper Tupac Shakur’s “performance” at Coachella 2012, but that wasn’t technically a hologram (it was a mirror-based optical illusion called “Pepper’s Ghost”).

A hologram is a flat image that appears to be three-dimensional.

Consider a standard photograph of an apple. No matter how you move your head or angle the photograph, it’s always going to look like the same flat image of an apple.

Flat pictures like this are created by recording the light waves reflecting off whatever is in a camera’s view when its shutter was clicked.

But a hologram of an apple would have depth. As you moved your head or the hologram itself, you’d feel like you were seeing new angles of the fruit.

Holograms use both the brightness and the phase of each light wave to give the viewer the sense that they’re looking at something three-dimensional — even though they aren’t.

A Hologram-Making AI

Traditionally, holograms were created using laser beams, but the images resulting from that technique could only be displayed as hard copies that were difficult to reproduce. The method couldn’t translate to video, either — only static images.

Computer-based techniques for making holograms can overcome those limitations, but they involve running physics-based simulations, which require a lot of processing power.

We are amazed at how well it performs.

Wojciech Matusik

It could take a supercomputer cluster minutes to generate a single hologram, and the final result still might not be photorealistic.

To speed up the process, while cutting down on the computational burden, MIT has developed a new AI-based technique they call “tensor holography.”

The researchers started by creating a training dataset of 4,000 computer-generated images and their matching photorealistic holograms. Each of the images included the color and depth information for every pixel.

They then trained an AI with this data, teaching it how to generate a hologram for almost any given 2D image. By the time it was finished, it could produce a photorealistic hologram in milliseconds.

“We are amazed at how well it performs,” researcher Wojciech Matusik told MIT News.

The AI requires less than 1 MB of memory — a fraction of what’s on most smartphones. The tech needed to calculate depth information also comes standard on many of today’s phones, meaning the devices could easily support the hologram-making process.

“It’s a considerable leap that could completely change people’s attitudes toward holography,” Matusik said. “We feel like neural networks were born for this task.”

We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].