MIT’s new AI can make holograms in real-time

It’s efficient enough to run on smartphones — no supercomputers necessary.
Sign up for the Freethink Weekly newsletter!
A collection of our favorite stories straight to your inbox

Holograms could make virtual reality more immersive, improve our 3D printers, and even help doctors diagnose and treat patients — if only they weren’t so difficult to create.

“It’s often been said that commercially available holographic displays will be around in 10 years, yet this statement has been around for decades,” MIT researcher Liang Shi told MIT News.

Now, Shi and his colleagues at MIT have developed a technique to generate holograms in real-time — and it’s so efficient, it could be done on a laptop or smartphone.

What Is a Hologram?

Pop culture has made the term “hologram” synonymous with deceased rapper Tupac Shakur’s “performance” at Coachella 2012, but that wasn’t technically a hologram (it was a mirror-based optical illusion called “Pepper’s Ghost”).

A hologram is a flat image that appears to be three-dimensional.

Consider a standard photograph of an apple. No matter how you move your head or angle the photograph, it’s always going to look like the same flat image of an apple.

Flat pictures like this are created by recording the light waves reflecting off whatever is in a camera’s view when its shutter was clicked.

But a hologram of an apple would have depth. As you moved your head or the hologram itself, you’d feel like you were seeing new angles of the fruit.

Holograms use both the brightness and the phase of each light wave to give the viewer the sense that they’re looking at something three-dimensional — even though they aren’t.

A Hologram-Making AI

Traditionally, holograms were created using laser beams, but the images resulting from that technique could only be displayed as hard copies that were difficult to reproduce. The method couldn’t translate to video, either — only static images.

Computer-based techniques for making holograms can overcome those limitations, but they involve running physics-based simulations, which require a lot of processing power.

We are amazed at how well it performs.


Wojciech Matusik

It could take a supercomputer cluster minutes to generate a single hologram, and the final result still might not be photorealistic.

To speed up the process, while cutting down on the computational burden, MIT has developed a new AI-based technique they call “tensor holography.”

The researchers started by creating a training dataset of 4,000 computer-generated images and their matching photorealistic holograms. Each of the images included the color and depth information for every pixel.

They then trained an AI with this data, teaching it how to generate a hologram for almost any given 2D image. By the time it was finished, it could produce a photorealistic hologram in milliseconds.

“We are amazed at how well it performs,” researcher Wojciech Matusik told MIT News.

The AI requires less than 1 MB of memory — a fraction of what’s on most smartphones. The tech needed to calculate depth information also comes standard on many of today’s phones, meaning the devices could easily support the hologram-making process.

“It’s a considerable leap that could completely change people’s attitudes toward holography,” Matusik said. “We feel like neural networks were born for this task.”

We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].

Sign up for the Freethink Weekly newsletter!
A collection of our favorite stories straight to your inbox
Related
AI skeptic Gary Marcus on AI’s moral and technical shortcomings
From hallucinations to regulatory battles, Gary Marcus argues the AI status quo has failed us and it’s time citizens demand something more.
Flexport is using generative AI to create the “holy grail” of shipping
Flexport is using generative AI to read documents, talk to truckers, and create a “knowledge agent” that’s an expert in shipping.
The West needs more water. This Nobel winner may have the answer.
Paul Migrom has an Emmy, a Nobel, and a successful company. There’s one more big problem on the to-do list.
Can we automate science? Sam Rodriques is already doing it.
People need to anticipate the revolution that’s coming in how humans and AI will collaborate to create discoveries, argues Sam Rodrigues.
AI is now designing chips for AI
AI-designed microchips have more power, lower cost, and are changing the tech landscape.
Up Next
underwater glider
Subscribe to Freethink for more great stories