AI device helps wheelchair users control their world

This smart device is helping people with disabilities overcome challenges like turning the light on, driving in reverse, or even finishing a sentence.

David Hojah always liked to invent things. Even as a kid, he spent his playtime dreaming up new gadgets. Becoming an engineer was a natural step. But it wasn’t until he heard a talk by Steve Saling that Hojah decided to direct his focus on helping people with disabilities.

Saling is an engineer and founder of the ALS Residence Initiative. In a wheelchair with ALS, Saling designed a smart home for people with disabilities like him, to improve their quality of life. Hojah says that learning about Saling’s work inspired him “to dive deep and help people with disabilities.”

So Hojah and a team of engineering students created Loro. Loro may look like a cartoonish pink bird perched on the back of the wheelchair, but it is actually a smart robot companion for disabled people. Paired with a tablet, it uses machine learning and eye-tracking to help people with limited mobility gain independence.

Loro, which means “parrot” in Spanish, is a 360 camera with artificial intelligence encased in an elegant pink bird. This device helps people with limited mobility gain independence by aiding with things like speech and typing. Photo courtesy of Loro.

Hojah and co-creators (four graduates of Harvard and MIT) started with the basics: navigation. Some wheelchair users can’t turn their head to look behind them, so the team used a pole-mounted, 360° camera (elegantly encased in a pink bird) to display the wheelchair’s surroundings on a tablet screen.

After talking with doctors, therapists, and people with disabilities, the team learned that sometimes, if an individual has a hard time speaking, just having a conversation can be an exhausting challenge. They then added AI communication technology, and Loro (meaning “parrot” in Spanish) got a brain, too. Paired with a tablet, the bird scans the user’s surroundings to suggest sentences on the tablet. For example, if Loro’s AI sees a bottle of water via the camera, it will recommend “I am thirsty,” as one option.

To round out its capabilities, Hojah’s team added eye-tracking technology. Typing is a slow process for anyone, but especially for those with mobility issues. Eye-tracking technology allows them to type, control the camera, and select AI sentences just by looking at the screen, and they can connect to smart home devices with the blink of an eye.

Hojah watched the man’s eyes look to different areas of the screen to control Loro and saw a tear welling up in his eye. Hojah says he realized then that his work really could benefit disabled people.

The product is due to be released later this year. Hojah says that he knew they were on the right track when he asked a friend to give it a try. Hojah watched the man’s eyes look to different areas of the screen to control Loro and saw a tear welling up in his eye. Hojah says he realized then that his work really could benefit disabled people.

“I saw that, and I felt very happy in my heart,” he says, adding that he thinks Loro will give wheelchair users more independence. No longer will they need to ask for help to turn the light on or off, drive their wheelchair in reverse, or finish a sentence. The smart device will do some of that work for them.

However, Jutta Treviranus, Director of the Inclusive Design Research Centre, says that assistive devices are usually not one-size-fits-all products. Treviranus does not work directly with Loro, but as a digital accessibility designer, she has experience working on similar technology. She says the most successful assistive devices are those that could also be useful to the masses – because of the cost-effectiveness of producing a product in high demand. For instance, the telephone, invented by Alexander Graham Bell for his hearing-impaired mother, and the audiobook, invented for the vision-impaired, were both imagined initially for people with disabilities.

Economies of scale pose a problem for companies like Loro and customers they serve — making it hard for people with disabilities to get the products they need.

Conversely, she says, small companies that make assistive devices for a niche market don’t survive. People with severe disabilities are diverse, and their needs are variable. The catch-22 facing designers is that the better tailored the invention is to very specific needs, the less likely it is to have a large customer base. Treviranus says the economies of scale pose a problem for companies like Loro and customers they serve — making it hard for people to get the products they need.

“Loro is a great step. The problem isn’t Loro. The problem is how we support things like Loro and how the markets work and how people with disabilities can afford things like Loro,” she says. “We need to provide them support. We also need to rethink how we mass produce things and how our markets work because our markets are abandoning everyone at the edge.”

Hojah recognizes the challenges his team will face as they launch their start-up, citing funding as the biggest hurdle. Still, he is optimistic. Already they have help from Harvard Innovation Labs, Verizon 5G Labs, and angel investors. And Hoja has plans for expanding its offerings. Loro is currently designed for people with speech or mobility issues, but he hopes future iterations will include support for people with cognitive problems like Alzheimer’s or autism. With Hoja and his team’s pioneering work bringing AI to assistive devices, taking Loro to the next level isn’t a far leap.

OpenAI’s GPT-4 outperforms doctors in another new study
OpenAI’s most powerful AI model, GPT-4, outperformed junior doctors in deciding how to treat patients with eye problems.
Watch the first AI vs. human dogfight using military jets
An AI fighter pilot faced off against a human pilot in a “dogfight” using actual planes — a huge milestone in military automation.
AI can help predict whether a patient will respond to specific tuberculosis treatments
Instead of a one-size-fits-all treatment approach, AI could help personalize treatments for each patient to provide the best outcomes.
New AI music generator makes songs from text prompts
AI music generators — AIs that create new music based on users’ text prompts — are lowering the bar for music creation, for better or worse.
Toward truly compostable plastic
Materials scientists are cooking up environmentally friendly plastics from natural sources like silk, plant fibers and whole algae.
Up Next
heart valve
Subscribe to Freethink for more great stories