AI device helps wheelchair users control their world

This smart device is helping people with disabilities overcome challenges like turning the light on, driving in reverse, or even finishing a sentence.

David Hojah always liked to invent things. Even as a kid, he spent his playtime dreaming up new gadgets. Becoming an engineer was a natural step. But it wasn’t until he heard a talk by Steve Saling that Hojah decided to direct his focus on helping people with disabilities.

Saling is an engineer and founder of the ALS Residence Initiative. In a wheelchair with ALS, Saling designed a smart home for people with disabilities like him, to improve their quality of life. Hojah says that learning about Saling’s work inspired him “to dive deep and help people with disabilities.”

So Hojah and a team of engineering students created Loro. Loro may look like a cartoonish pink bird perched on the back of the wheelchair, but it is actually a smart robot companion for disabled people. Paired with a tablet, it uses machine learning and eye-tracking to help people with limited mobility gain independence.

Loro, which means “parrot” in Spanish, is a 360 camera with artificial intelligence encased in an elegant pink bird. This device helps people with limited mobility gain independence by aiding with things like speech and typing. Photo courtesy of Loro.

Hojah and co-creators (four graduates of Harvard and MIT) started with the basics: navigation. Some wheelchair users can’t turn their head to look behind them, so the team used a pole-mounted, 360° camera (elegantly encased in a pink bird) to display the wheelchair’s surroundings on a tablet screen.

After talking with doctors, therapists, and people with disabilities, the team learned that sometimes, if an individual has a hard time speaking, just having a conversation can be an exhausting challenge. They then added AI communication technology, and Loro (meaning “parrot” in Spanish) got a brain, too. Paired with a tablet, the bird scans the user’s surroundings to suggest sentences on the tablet. For example, if Loro’s AI sees a bottle of water via the camera, it will recommend “I am thirsty,” as one option.

To round out its capabilities, Hojah’s team added eye-tracking technology. Typing is a slow process for anyone, but especially for those with mobility issues. Eye-tracking technology allows them to type, control the camera, and select AI sentences just by looking at the screen, and they can connect to smart home devices with the blink of an eye.

Hojah watched the man’s eyes look to different areas of the screen to control Loro and saw a tear welling up in his eye. Hojah says he realized then that his work really could benefit disabled people.

The product is due to be released later this year. Hojah says that he knew they were on the right track when he asked a friend to give it a try. Hojah watched the man’s eyes look to different areas of the screen to control Loro and saw a tear welling up in his eye. Hojah says he realized then that his work really could benefit disabled people.

“I saw that, and I felt very happy in my heart,” he says, adding that he thinks Loro will give wheelchair users more independence. No longer will they need to ask for help to turn the light on or off, drive their wheelchair in reverse, or finish a sentence. The smart device will do some of that work for them.

However, Jutta Treviranus, Director of the Inclusive Design Research Centre, says that assistive devices are usually not one-size-fits-all products. Treviranus does not work directly with Loro, but as a digital accessibility designer, she has experience working on similar technology. She says the most successful assistive devices are those that could also be useful to the masses – because of the cost-effectiveness of producing a product in high demand. For instance, the telephone, invented by Alexander Graham Bell for his hearing-impaired mother, and the audiobook, invented for the vision-impaired, were both imagined initially for people with disabilities.

Economies of scale pose a problem for companies like Loro and customers they serve — making it hard for people with disabilities to get the products they need.

Conversely, she says, small companies that make assistive devices for a niche market don’t survive. People with severe disabilities are diverse, and their needs are variable. The catch-22 facing designers is that the better tailored the invention is to very specific needs, the less likely it is to have a large customer base. Treviranus says the economies of scale pose a problem for companies like Loro and customers they serve — making it hard for people to get the products they need.

“Loro is a great step. The problem isn’t Loro. The problem is how we support things like Loro and how the markets work and how people with disabilities can afford things like Loro,” she says. “We need to provide them support. We also need to rethink how we mass produce things and how our markets work because our markets are abandoning everyone at the edge.”

Hojah recognizes the challenges his team will face as they launch their start-up, citing funding as the biggest hurdle. Still, he is optimistic. Already they have help from Harvard Innovation Labs, Verizon 5G Labs, and angel investors. And Hoja has plans for expanding its offerings. Loro is currently designed for people with speech or mobility issues, but he hopes future iterations will include support for people with cognitive problems like Alzheimer’s or autism. With Hoja and his team’s pioneering work bringing AI to assistive devices, taking Loro to the next level isn’t a far leap.

AI narrates 5,000 free audiobooks for Project Gutenberg
A new text-to-speech system developed by Microsoft and MIT was used to create nearly 5,000 audiobooks for Project Gutenberg.
Why Toyota is building a “kindergarten for robots”
Toyota is using a generative AI-based method to teach robots to peel veggies, prepare snacks, and perform other dexterous tasks.
UT med students can now get a dual degree in AI
The University of Texas at San Antonio has launched what it says is the US’s first dual degree in medicine and AI.
New low-carbon cement is stronger than the regular stuff
A low-carbon cement developed by two MIT grads has officially exceeded industry standards for strength, durability, and more.
Self-driving cars can now tell passengers what they’re thinking
AV startup Wayve has given its self-driving cars the ability to explain their decisions in conversational language.
Up Next
heart valve
Subscribe to Freethink for more great stories