Meta’s first-of-its-kind AI can translate between any of 100 languages

The company is one step closer to its goal of creating a universal translator.

Meta has unveiled a first-of-its-kind AI that can translate up to 100 languages, putting the company a step closer to its goal of creating a universal translator.

The language barrier: An estimated 7,000 languages are currently spoken across the globe, but 87% of people know just one or two of them.

Even if you know the most popular language — English — it’s only spoken by 18% of the world, meaning there is still a language barrier between you and the vast majority of people.

This barrier can do more than just make it hard for you to chat with others or order food on vacation. If you don’t speak the local language, you can have trouble getting around, finding and keeping a job, or accessing quality healthcare and education

“The goal here is instantaneous speech-to-speech translation across all languages.”

Mark Zuckerberg

A universal translator: Science fiction has already come up with a solution to the language barrier problem. Dubbed a “universal translator,” it’s a device capable of instantly translating anything written or spoken in one language into text or speech in any other.

Meta is on a mission to make such a device for real, with the help of AI.

“The goal here is instantaneous speech-to-speech translation across all languages, even those that are mostly spoken; the ability to communicate with anyone in any language,” said CEO Mark Zuckerberg in 2022. “That’s a superpower that people dreamed of forever and AI is going to deliver that within our lifetimes.”

How it works: While there are already translation apps, they aren’t able to translate every language like sci-fi’s universal translator, and the translations they produce often contain major errors

This inaccuracy, according to Meta, is partially due to how current translation tech takes a “cascading” approach that requires the use of multiple subsystems.

To translate spoken Spanish into spoken French, for example, the tech might use a speech recognition system to transcribe the Spanish audio into text, a text-to-text translation system to translate that into French, and then a text-to-speech system to turn that into French audio.

“SeamlessM4T’s single system approach reduces errors and delays.”

Meta

Meta’s newly announced translation tool, SeamlessM4T, can recognize audio or text in nearly 100 languages and translate it into text in any of those languages or speech in 36 languages, including English. And unlike other translation tools, it relies on just one AI model to complete all of these tasks.

According to Meta, this makes it “the first all-in-one multilingual multimodal AI translation and transcription model,” and it led to greater accuracy and faster translation.

“Compared to approaches using separate models, SeamlessM4T’s single system approach reduces errors and delays, increasing the efficiency and quality of the translation process,” Meta researchers wrote in a white paper.

The cold water: SeamlessM4T may be an improvement on existing systems, but it’s still not 100% accurate and is limited to just a fraction of the many languages spoken worldwide, meaning Meta may be closer to achieving the dream of a universal translator, but it isn’t there just yet.

The white paper also notes that SeamlessM4T exhibited some signs of gender bias in its translations — in languages with grammatical genders, it might offer the male form of the word “doctor,” for example, even when the context of the sentence makes it clear that the doctor is a woman. This is because the data used to train the AI wasn’t free of gender bias — better training data and filters could help alleviate that issue.

Additionally, mistakes made by SeamlessM4T introduced “toxic elements” into about 0.16% of translations. Though Meta says that is less than existing translation tech, it’s still an area that could be improved.

“This is only the latest step in our ongoing effort to build AI-powered technology that helps connect people across languages.”

Meta

Looking ahead: Meta hasn’t announced any plans to turn SeamlessM4T into a commercial product, but it has released a demo tool that allows users to say something in one language and have it translated into speech and text in up to three others.

It has also made the AI model and its code available online so that other researchers can build on the tech.

“This is only the latest step in our ongoing effort to build AI-powered technology that helps connect people across languages,” wrote Meta in a blog post. “In the future, we want to explore how this foundational model can enable new communication capabilities — ultimately bringing us closer to a world where everyone can be understood.”

We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].

Related
ChatGPT forces us to ask: how much of “being human” belongs to us?
Large language models have been trained on massive amounts of “natural” human language — just like us. Does this make the robots part human?
You can now talk to ChatGPT and show it pictures
OpenAI is rolling out new features that let subscribers talk to ChatGPT and show it pictures, enabling more intuitive interactions.
AIs accurately predicted path of Hurricane Lee a week out
AI-based weather forecasting models developed by Google, Nvidia, and Huawei accurately predicted where Hurricane Lee would make landfall.
AI narrates 5,000 free audiobooks for Project Gutenberg
A new text-to-speech system developed by Microsoft and MIT was used to create nearly 5,000 audiobooks for Project Gutenberg.
Why Toyota is building a “kindergarten for robots”
Toyota is using a generative AI-based method to teach robots to peel veggies, prepare snacks, and perform other dexterous tasks.
Up Next
A humanoid robot is moving boxes in a warehouse.
Subscribe to Freethink for more great stories