Making VR more accessible for the hearing impaired

In his latest project, Myles de Bastion shows how VR could help the hearing-impaired in remote settings.

Virtual tools became a crucial part of work during the pandemic — but for those with hearing impairment, these options made navigating work environments all the more difficult.

In “Virtual Worlds Made Accessible Beyond Sound,” deaf artist, musician, and designer Myles de Bastion reveals how VR could address this problem — and what is still needed for the medium to realize this inclusive potential.

The challenge: There are alternatives to audio meetings, but they aren’t great. Auto-captioning tends to lag and isn’t always accurate (Zoom didn’t even offer auto-captioning until November 2020), while video calls often lack the spatial context often needed for communication in sign language.

Key quote: “In a Zoom meeting you don’t have any sense of physical space, so you lose a lot of the ability to communicate (in American Sign Language) fluently, and conversations take much longer. VR solves this by restoring the sense of physical space that is needed for ASL,” Myles de Bastion said in his presentation at Open Signal in Portland.

On display: In the project, de Bastion examines how virtual environments in a VR headset could address these core problems with sign language and what he calls “environmental captioning.”

Context: VR hasn’t always been conducive for sign language because it traditionally relied on controllers. Oculus announced hand tracking on its Quest headset in Fall 2019: with this feature, users can express themselves through their hands and fingers.

  • Buuuut: Headset-based hand tracking still isn’t perfect, and it is still not available on many applications.

  • Hands are perceived through cameras located on the headset itself. When one hand blocks another from view, for example, meaning can be lost.

Myles is leading the way in defining accessibility as a norm by developing these tools to create more inclusive virtual worlds.

Sonya Neunzert

Caption this: In the presentation, de Bastion shows how environmental captioning could literally meet users where they are, rather than forcing them to only look in one place for all captioning. He shows how dialogue can be linked to VR avatars, without obstructing others’ views of them, and encourages other designers to think about coloring, opacity, and shapes that change relative to other users’ proximities.

Just the beginning: De Bastion advocated for more inclusion in the extended reality (XR) industry, writing that “involving (d)eaf & disabled (people) will increase diversity of ideas in virtual spaces … (and accelerate the) mass adoption of VR.”

New norms: “One of the most interesting things to me about immersive tech is seeing people who actively define the norms and protocols in these new kinds of media that are still basically being invented,” de Bastion’s Mentor Sonya Neunzert said in a statement. “Myles is leading the way in defining accessibility as a norm by developing these tools to create more inclusive virtual worlds.”

We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].

VR exposure therapy app lets you face phobias on your own terms 
Exposure therapy is a reliable technique for overcoming irrational fears, but it currently suffers high patient dropout rates.
Google has not created sentient AI — yet 
Google can make AI seem sentient, but the AI has no understanding of what it is saying. But humans are easily fooled by conversational AI.
Augmented reality will give us superpowers
Virtual and augmented reality have had false starts, but AR eyewear will soon replace the smartphone as our interface for digital content.
Metaverse real estate prices are booming. This is why. 
People are paying millions of dollars to buy plots of pixellated land in virtual worlds, even though they doesn’t fully exist yet.
4 things you need to know about the metaverse 
Technology that merges the virtual and physical worlds could start to become a reality for consumers in just a few years.
Up Next
rec room
Subscribe to Freethink for more great stories