Skip to main content
Move the World.
augmented reality glasses

Lead image courtesy of Facebook.

Don't be fooled by Facebook's augmented reality glasses. Despite being "glasses," the experience is entirely auditory.

Imagine dining at a cafe, and sounds of clinking silverware disappear. You glance across the table at your friend, and their voice becomes more audible as the chatter coming from the adjacent table fades away. You hear a mountain river and caws of tropical birds — despite sitting in a Starbucks in the middle of the city.

Augmented reality blends the real and virtual worlds by superimposing computer-generated sounds and images in the real world.

Facebook Reality Labs has now released details of how they plan to make augmented reality glasses that can tune out the real world and make computer-generated sounds seem like they're coming from your actual surroundings. And they are close — but not quite there.

Contextually-Aware Noise Cancellation

The glasses look like regular earbuds, which can be attached to a headset with other devices. Facebook hopes they will enhance face-to-face conversations by erasing extraneous noise in the room and intensifying only the sounds you want to hear. They do this with contextually-aware noise cancellation. Unlike noise cancellation that we are most familiar with, which reduces all outside noise, contextually-aware noise cancellation highlights sounds you want to hear and dims the rest.

Facebook hopes the glasses will enhance face-to-face conversations by erasing extraneous noise in the room and intensifying only the sounds you want to hear.

They can do this by using multiple tiny microphones that collect sound in your vicinity. Then, by gleaning information from your head movements, it filters out the sounds you aren't interested in, homing in on those that have your attention.

Audio Presence

The audio team from FRL Research is also creating "Audio Presence," which would replicate how sounds travel in real space — essentially making virtual sounds coming from the augmented reality glasses indistinguishable from the real world. The goal is that it will be as good as listening to the world around you, instead of an electronic device, reports Engadget.

"Imagine if you were on a phone call and you forgot that you were separated by distance," Research Lead Philip Robinson told Road to VR. "That's the promise of the technology we're developing."

Sensors on the headset, combined with AI, can map the space's geometry and estimate its acoustic properties, giving the headset a general idea of how sound propagates through the room. It can then apply that insight to virtual sounds, making them seem as if they are also coming from somewhere in the area, not the headset.

The crux is in creating a custom "Head-related Transfer Function" (HRTF). It is a digital version of an individual's ear and how it hears a sound. They are costly and challenging to make, but Facebook hopes to be soon able to generate an HRTF from a photo of an ear, reports Road to VR — yet one more thing Facebook can figure out from a selfie.

The State of Audio AR

Improving the AR audio experience is something that Google, Apple, and other big tech companies are also working on.

Imagine a Zoom meeting where everyone is sitting around a table, and you can hear people to your left, right, and ahead. Apple has a patent for headphones that can do just that. They can identify where sounds come from, enabling users to hear where people are located, even when they are not physically present. And rumors of Apple's augmented reality glasses or headset leaked earlier this summer. At about the same time, Google announced an AR update that would combine 3D environments and insert virtual objects more realistically, while Microsoft announced new AR tools for its HoloLens.

If all this sounds straight out of a spy movie — smart glasses that pick up a conversation across the room — there is no need to worry yet.

Chief scientist Michael Abrash at FRL Research told The Verge that the technology isn't powerful enough for long-range spying. But: "If you ever see anyone walking with glasses that are two feet wide... you should be suspicious."

But, as with any technology once developed, they always seem to get smaller and more inconspicuous.

We'd love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected]

Up Next

Through the Looking Glass
Nature is Good for You. What About VR Nature?
VR Nature
Through the Looking Glass
Nature is Good for You. What About VR Nature?
Nature has the power to reduce stress and enhance our moods. Can VR nature experiences be a substitute for physically spending time in the outdoors?

Nature has the power to reduce stress and enhance our moods. Can VR nature experiences be a substitute for physically spending time in the outdoors?

Challengers
How VR Could Change Your Life
How VR Could Change Your Life
Watch Now
Challengers
How VR Could Change Your Life
Virtual reality could alter the human experience forever.
Watch Now

Imagine if you could climb Mount Everest and go on stage at a Beyonce concert with your friends...before breakfast. Linc Gasking and his team of visual effects experts at 8i thinks they can make holograms of humans so real, that VR will go mainstream and they’ll alter the human experience forever.

Animals
Scientists Put Bees in Little Reflective “Vests” for Bee Tracking
bee tracking
Animals
Scientists Put Bees in Little Reflective “Vests” for Bee Tracking
Bee tracking is difficult and expensive, and smaller bees are left out. Lightweight, inexpensive tags may be a solution.

Bee tracking is difficult and expensive, and smaller bees are left out. Lightweight, inexpensive tags may be a solution.

Medical Innovations
A New Way to Heal Badly Broken Bones with Electrical Stimulation
electrical stimulation
Medical Innovations
A New Way to Heal Badly Broken Bones with Electrical Stimulation
A new bone scaffold delivers electrical stimulation to broken bones, speeds up the healing process, and then simply dissolves into the body.

A new bone scaffold delivers electrical stimulation to broken bones, speeds up the healing process, and then simply dissolves into the body.

Solar Energy
24/7 Solar Power is On The Horizon
solar power
Solar Energy
24/7 Solar Power is On The Horizon
Researchers at the Air Force Research Laboratory are planning new solar power tech that collects energy in outer space and laser-beams it back to Earth.

Researchers at the Air Force Research Laboratory are planning new solar power tech that collects energy in outer space and laser-beams it back to Earth.

INTEL
Bringing Virtual Reality to Brain Surgery
Bringing Virtual Reality to Brain Surgery
Watch Now
INTEL
Bringing Virtual Reality to Brain Surgery
Virtual reality is helping surgeons and patients prepare for complicated, life-saving surgeries in ways never before possible.
Watch Now

Brain surgery is never easy -- for the doctor or the patient. Now, virtual reality is changing the game. Surgical Theater has created a revolutionary new tool, powered by Intel technology, that allows surgeons and patients to prepare for complicated new surgeries in ways never before possible. Surgeons have previously had to rely on 2D images and their imagination to visualize a surgery, but now they are able to use 3D, VR...

AI
This Computer Can Write 2,000 Snarky Articles Per Second
This Computer Can Write 2,000 Snarky Articles Per Second
AI
This Computer Can Write 2,000 Snarky Articles Per Second
What does it mean for the future of journalism when a computer can turn mounds of data into a cohesive narrative?
By Mike Riggs

What does it mean for the future of journalism when a computer can turn mounds of data into a cohesive narrative?