Skip to main content
Move the World.
augmented reality glasses

Lead image courtesy of Facebook.

Don't be fooled by Facebook's augmented reality glasses. Despite being "glasses," the experience is entirely auditory.

Imagine dining at a cafe, and sounds of clinking silverware disappear. You glance across the table at your friend, and their voice becomes more audible as the chatter coming from the adjacent table fades away. You hear a mountain river and caws of tropical birds — despite sitting in a Starbucks in the middle of the city.

Augmented reality blends the real and virtual worlds by superimposing computer-generated sounds and images in the real world.

Facebook Reality Labs has now released details of how they plan to make augmented reality glasses that can tune out the real world and make computer-generated sounds seem like they're coming from your actual surroundings. And they are close — but not quite there.

Contextually-Aware Noise Cancellation

The glasses look like regular earbuds, which can be attached to a headset with other devices. Facebook hopes they will enhance face-to-face conversations by erasing extraneous noise in the room and intensifying only the sounds you want to hear. They do this with contextually-aware noise cancellation. Unlike noise cancellation that we are most familiar with, which reduces all outside noise, contextually-aware noise cancellation highlights sounds you want to hear and dims the rest.

Facebook hopes the glasses will enhance face-to-face conversations by erasing extraneous noise in the room and intensifying only the sounds you want to hear.

They can do this by using multiple tiny microphones that collect sound in your vicinity. Then, by gleaning information from your head movements, it filters out the sounds you aren't interested in, homing in on those that have your attention.

Audio Presence

The audio team from FRL Research is also creating "Audio Presence," which would replicate how sounds travel in real space — essentially making virtual sounds coming from the augmented reality glasses indistinguishable from the real world. The goal is that it will be as good as listening to the world around you, instead of an electronic device, reports Engadget.

"Imagine if you were on a phone call and you forgot that you were separated by distance," Research Lead Philip Robinson told Road to VR. "That's the promise of the technology we're developing."

Sensors on the headset, combined with AI, can map the space's geometry and estimate its acoustic properties, giving the headset a general idea of how sound propagates through the room. It can then apply that insight to virtual sounds, making them seem as if they are also coming from somewhere in the area, not the headset.

The crux is in creating a custom "Head-related Transfer Function" (HRTF). It is a digital version of an individual's ear and how it hears a sound. They are costly and challenging to make, but Facebook hopes to be soon able to generate an HRTF from a photo of an ear, reports Road to VR — yet one more thing Facebook can figure out from a selfie.

The State of Audio AR

Improving the AR audio experience is something that Google, Apple, and other big tech companies are also working on.

Imagine a Zoom meeting where everyone is sitting around a table, and you can hear people to your left, right, and ahead. Apple has a patent for headphones that can do just that. They can identify where sounds come from, enabling users to hear where people are located, even when they are not physically present. And rumors of Apple's augmented reality glasses or headset leaked earlier this summer. At about the same time, Google announced an AR update that would combine 3D environments and insert virtual objects more realistically, while Microsoft announced new AR tools for its HoloLens.

If all this sounds straight out of a spy movie — smart glasses that pick up a conversation across the room — there is no need to worry yet.

Chief scientist Michael Abrash at FRL Research told The Verge that the technology isn't powerful enough for long-range spying. But: "If you ever see anyone walking with glasses that are two feet wide... you should be suspicious."

But, as with any technology once developed, they always seem to get smaller and more inconspicuous.

We'd love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected]

Up Next

Through the Looking Glass
Nature is Good for You. What About VR Nature?
VR Nature
Through the Looking Glass
Nature is Good for You. What About VR Nature?
Nature has the power to reduce stress and enhance our moods. Can VR nature experiences be a substitute for physically spending time in the outdoors?

Nature has the power to reduce stress and enhance our moods. Can VR nature experiences be a substitute for physically spending time in the outdoors?

Challengers
How VR Could Change Your Life
How VR Could Change Your Life
Watch Now
Challengers
How VR Could Change Your Life
Virtual reality could alter the human experience forever.
Watch Now

Imagine if you could climb Mount Everest and go on stage at a Beyonce concert with your friends...before breakfast. Linc Gasking and his team of visual effects experts at 8i thinks they can make holograms of humans so real, that VR will go mainstream and they’ll alter the human experience forever.

3D Printing
This May Be The 3D-Printed Submarine of the Future
3d printed submarine
3D Printing
This May Be The 3D-Printed Submarine of the Future
Using unique construction techniques, this 3D-printed submarine may set the pace for the building of large AUVs to come.

Using unique construction techniques, this 3D-printed submarine may set the pace for the building of large AUVs to come.

Artificial Intelligence
Training a Home Robot To See — and Hear
Home Robot
Artificial Intelligence
Training a Home Robot To See — and Hear
Facebook's AI lab has released new tools for its embodied AI training platform, including one to train a home robot to respond to sounds.

Facebook's AI lab has released new tools for its embodied AI training platform, including one to train a home robot to respond to sounds.

Climate Change
Raising Pacific Islands to Save Them From High Sea Levels
sea level rise
Climate Change
Raising Pacific Islands to Save Them From High Sea Levels
The president of Kiribati announced a new plan to fight against sea level rise: raise the islands.

The president of Kiribati announced a new plan to fight against sea level rise: raise the islands.

Health
“Electronic Nose” Can Detect a Cancer Precursor on Patients’ Breath
electronic nose
Health
“Electronic Nose” Can Detect a Cancer Precursor on Patients’ Breath
A new study found that an electronic nose could be an improved screening tool for Barrett’s esophagus, a precursor to esophageal cancer.

A new study found that an electronic nose could be an improved screening tool for Barrett’s esophagus, a precursor to esophageal cancer.

Dispatches
The Cause (and Possible Cure) for Most Infertility
The Cause (and Possible Cure) for Most Infertility
Dispatches
The Cause (and Possible Cure) for Most Infertility
Fertility medicine may be on the edge of a breakthrough.

Fertility medicine may be on the edge of a breakthrough.