Startup breaks through “accent barrier” with real-time translator

The accent translator can integrate into Zoom, WhatsApp, or phone calls.

After struggling to understand each other’s accents, three Stanford students — from China, Russia, and Venezuela — developed a technology that can listen to English spoken with one accent and replay it with another.

They’ve now formed a startup, Sanas, to release the tech, which they say is the world’s first real-time speech accent translator.

The challenge: Of the 1.5 billion people who know English, more than 1 billion speak it as a second language. Those who speak it as a first language hail from the U.S., the U.K., Ireland, Australia, and other regions with their own unique pronunciations of English words.

Given all of that, it’s easy to see how two people can both be speaking English and still have difficulty communicating due to accents, from either their home region or their home language.

“We knew from our own experience that forcing a different accent on yourself is uncomfortable.”

Andres Perez Soderi

Speech therapy can help non-native speakers lose their accents, but it takes a long time and doesn’t work for everyone, and some people would rather not “fake” a local accent.

“[W]e knew from our own experience that forcing a different accent on yourself is uncomfortable,” Sanas CFO Andres Perez Soderi told IEEE Spectrum. “I went to a British high school and tried to force a British accent; it was an experience that was hard to digest.”

How it works: Rather than trying to change how people speak, the students decided to train an accent translator algorithm. First, they had to feed it a lot of recordings of the exact same phrases spoken with different accents.

“You aren’t just doing audio signal processing, changing the pitch and tone — you have to change the phonetics,” Sanas CTO Shawn Zhang explained.

“So we really needed parallel data sets, created by readers using the same source material, so the neural network could learn to map from one to the other, examining both to learn how to transform the pronunciation,” he continued.

accent translator
A preview of what users see when using the accent translator. Credit: Sanas

Their finished accent translator works for five accents: American, British, Australian, Filipino, and Spanish — you could say something in Spanish-accented English, for example, and have it translated into a British accent.

It has a 150-millisecond delay (about one-sixth of a second), runs directly on a person’s computer (not in the cloud), and can integrate into apps such as  Zoom and WhatsApp. 

The total delay experienced while using the tech depends on the app you’re communicating with — Zoom, for example, averages a 50-millisecond delay itself, so someone using the accent translator with that service would experience a total delay of 200 milliseconds.

However, Soderi told IEEE Spectrum that anything below 300 milliseconds is generally imperceptible.

It could be a boon to businesses that provide customer service and tech support over the phone.

The next steps: Sanas has secured $5.5 million in funding, which the students will use to expand their team and further develop the tech. In addition to adding other accents within English, they plan to start translating other languages, too (Spanish spoken in various accents, for example).

While the students’ personal lives may have inspired them to develop the accent translator, they think it could be a boon to many businesses, particularly those that provide customer service and technical support over the phone — they already have seven such companies piloting the tech.

“There are also creative use cases such as those in entertainment and media where producers can make their films and programs understandable in different parts of the world by matching accents to localities,” Sanas CEO Maxim Serebryakov said.

“We are also exploring how machines can better interpret what people are saying,” he continued. “We’ve only begun to explore the possibilities.”

We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].

You win this street game by getting “hit” by a self-driving car
A game that challenges pedestrians to avoid detection by an AI could help train tomorrow’s self-driving cars.
Breakthrough AI learns language with “human-like” efficiency
An new AI model learned to generalize language concepts with “human-like” efficiency and remix them to respond to novel inquires.
OpenAI launches a GPT Store for customized AI chatbots
ChatGPT users will soon be able to create, share, and potentially monetize customized versions of the chatbot through OpenAI’s GPT Store.
This AI companion could forever change how you feel about tech
Silicon Valley tech startup New Computer has unveiled Dot, an AI companion that could forever change our relationship with tech.
How AI played an instrumental role in making mRNA vaccines
Years before Moderna created an effective mRNA vaccine against COVID, the company put into place AI systems to accelerate the research process.
Up Next
Subscribe to Freethink for more great stories