Skip to main content
Move the World.
augmented reality glasses

Lead image courtesy of Facebook.

Don't be fooled by Facebook's augmented reality glasses. Despite being "glasses," the experience is entirely auditory.

Imagine dining at a cafe, and sounds of clinking silverware disappear. You glance across the table at your friend, and their voice becomes more audible as the chatter coming from the adjacent table fades away. You hear a mountain river and caws of tropical birds — despite sitting in a Starbucks in the middle of the city.

Augmented reality blends the real and virtual worlds by superimposing computer-generated sounds and images in the real world.

Facebook Reality Labs has now released details of how they plan to make augmented reality glasses that can tune out the real world and make computer-generated sounds seem like they're coming from your actual surroundings. And they are close — but not quite there.

Contextually-Aware Noise Cancellation

The glasses look like regular earbuds, which can be attached to a headset with other devices. Facebook hopes they will enhance face-to-face conversations by erasing extraneous noise in the room and intensifying only the sounds you want to hear. They do this with contextually-aware noise cancellation. Unlike noise cancellation that we are most familiar with, which reduces all outside noise, contextually-aware noise cancellation highlights sounds you want to hear and dims the rest.

Facebook hopes the glasses will enhance face-to-face conversations by erasing extraneous noise in the room and intensifying only the sounds you want to hear.

They can do this by using multiple tiny microphones that collect sound in your vicinity. Then, by gleaning information from your head movements, it filters out the sounds you aren't interested in, homing in on those that have your attention.

Audio Presence

The audio team from FRL Research is also creating "Audio Presence," which would replicate how sounds travel in real space — essentially making virtual sounds coming from the augmented reality glasses indistinguishable from the real world. The goal is that it will be as good as listening to the world around you, instead of an electronic device, reports Engadget.

"Imagine if you were on a phone call and you forgot that you were separated by distance," Research Lead Philip Robinson told Road to VR. "That's the promise of the technology we're developing."

Sensors on the headset, combined with AI, can map the space's geometry and estimate its acoustic properties, giving the headset a general idea of how sound propagates through the room. It can then apply that insight to virtual sounds, making them seem as if they are also coming from somewhere in the area, not the headset.

The crux is in creating a custom "Head-related Transfer Function" (HRTF). It is a digital version of an individual's ear and how it hears a sound. They are costly and challenging to make, but Facebook hopes to be soon able to generate an HRTF from a photo of an ear, reports Road to VR — yet one more thing Facebook can figure out from a selfie.

The State of Audio AR

Improving the AR audio experience is something that Google, Apple, and other big tech companies are also working on.

Imagine a Zoom meeting where everyone is sitting around a table, and you can hear people to your left, right, and ahead. Apple has a patent for headphones that can do just that. They can identify where sounds come from, enabling users to hear where people are located, even when they are not physically present. And rumors of Apple's augmented reality glasses or headset leaked earlier this summer. At about the same time, Google announced an AR update that would combine 3D environments and insert virtual objects more realistically, while Microsoft announced new AR tools for its HoloLens.

If all this sounds straight out of a spy movie — smart glasses that pick up a conversation across the room — there is no need to worry yet.

Chief scientist Michael Abrash at FRL Research told The Verge that the technology isn't powerful enough for long-range spying. But: "If you ever see anyone walking with glasses that are two feet wide... you should be suspicious."

But, as with any technology once developed, they always seem to get smaller and more inconspicuous.

We'd love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected]

Up Next

Through the Looking Glass
Nature is Good for You. What About VR Nature?
VR Nature
Through the Looking Glass
Nature is Good for You. What About VR Nature?
Nature has the power to reduce stress and enhance our moods. Can VR nature experiences be a substitute for physically spending time in the outdoors?

Nature has the power to reduce stress and enhance our moods. Can VR nature experiences be a substitute for physically spending time in the outdoors?

Challengers
How VR Could Change Your Life
How VR Could Change Your Life
Watch Now
Challengers
How VR Could Change Your Life
Virtual reality could alter the human experience forever.
Watch Now

Imagine if you could climb Mount Everest and go on stage at a Beyonce concert with your friends...before breakfast. Linc Gasking and his team of visual effects experts at 8i thinks they can make holograms of humans so real, that VR will go mainstream and they’ll alter the human experience forever.

Wildfires
This Model May Help Predict Wildfires
predict wildfires
Wildfires
This Model May Help Predict Wildfires
Wildfires need dry fuel to thrive. Researchers at Stanford are working on an AI model to measure dryness to help predict, and control, the blaze.

Wildfires need dry fuel to thrive. Researchers at Stanford are working on an AI model to measure dryness to help predict, and control, the blaze.

Future of Medicine
Stem Cell Research Breakthrough Opens Path to Growing Human Organs in Animals
Stem Cell Research
Future of Medicine
Stem Cell Research Breakthrough Opens Path to Growing Human Organs in Animals
New stem cell research has revealed a way to coax human cells to grow to maturity in mouse models, a major advance in the field.

New stem cell research has revealed a way to coax human cells to grow to maturity in mouse models, a major advance in the field.

Dope Science
Microdosing Psychedelics
microdosing psychedelics
Watch Now
Dope Science
Microdosing Psychedelics
Microdosing is the no-flowers-in-your-hair practice of regularly taking minimal amounts of hallucinogenic substances. Although it’s currently illegal, millions of dollars are being poured into further research.
Watch Now

Microdosing is the no-flowers-in-your-hair practice of regularly taking minimal amounts of hallucinogenic substances. Although it’s currently illegal, millions of dollars are being poured into further research.

Uprising
What's Special About Cancer-Killing Nanobots? Precision.
medical nanobots
Uprising
What's Special About Cancer-Killing Nanobots? Precision.
These tiny, robotic machines can deliver drugs directly to infected cells, and they're changing the future of medicine.

These tiny, robotic machines can deliver drugs directly to infected cells, and they're changing the future of medicine.

Dispatches
Zika Could Be a "Smart Missile" for Brain Cancer
Zika Could Be a
Dispatches
Zika Could Be a "Smart Missile" for Brain Cancer
Zika can devastate fetal brains; scientists want to turn it against brain tumors instead.

Zika can devastate fetal brains; scientists want to turn it against brain tumors instead.