By Jasmin Sykes
A Kenyan entrepreneur who designed an app that translates speech into sign language using AI-powered 3D avatars has won a major award for African innovation.
“Think of it as Google Translate for sign language, that takes speech and text input and outputs photorealistic avatars signing,” said Elly Savatia, creator of Terp 360, a web-based app that uses 3D avatars to provide sign language translations from speech in real time, allowing users to communicate without relying on human interpreters.
Savatia was awarded £50,000 ($67,000) on October 16 by the UK’s Royal Academy of Engineering for winning its prestigious Africa Prize for Engineering Innovation, which recognizes entrepreneurs who have developed technology to address challenges on the continent. His innovation was selected from four finalists across Africa by a panel of seven judges, during a ceremony held in Dakar, Senegal.
“The Africa Prize for Engineering Innovation is a way for us to support, showcase and celebrate African innovation,” said Rebecca Enonchong, chair of the prize-judging panel.
“What really stood out about Elly’s solution, and Elly himself, is the level of innovation,” she said. “It was really a demonstration that Africans are able and capable of using cutting-edge technology to solve problems, not just on the continent but beyond,” she added.
Savatia explains that deaf people in Kenya often struggle to access essential services because many people working in those services don’t know sign language.
“To go to the workplace, education, health care, you have to communicate,” said Savatia. “But the deaf community, they’re left behind.”
He says human sign language interpreters are expensive and a shortage of interpreters means many deaf people across Africa fail to transition to higher education
Despite a government bill earlier this year requiring Kenyan employers to reserve at least 5% of jobs for people with disabilities, Savatia says many roles often remain inaccessible to deaf people.
“Companies cannot afford interpreters… [and] they just don’t have the tools to effectively integrate these people,” he says. “We see ourselves as an enabler. We are able to do sign language, but at scale,” he adds.
Terp 360 was developed in conjunction with deaf and hard-of-hearing Kenyans to record over 2,300 signs, including commonly used phrases and words. Motion sensors were attached to the hands of a signer, capturing the movements of their hands in space.
While other avatar-based sign language translation technologies already exist, Savatia says these are not designed with African sign languages or cultural context in mind.
There are more than 300 sign languages in use worldwide, and around 30 in Africa. Terp 360 currently translates from English and Swahili into Kenyan Sign Language. Savatia says the app is being developed to translate from other major African and global languages, and is on track to support Rwandan, Ugandan, South African, British and American sign languages by mid-2027.
To train the AI in new languages, the Terp 360 developers plan to partner with local NGOs and projects that have visual sign language data sets, as well as news stations with several years’ worth of sign language video. Savatia says they have also set up a motion capture studio in Nairobi, Kenya, with the capacity to record and learn 1,000 words a day.
“One of the criteria that we use to judge is social impact,” said Enonchong. “We want to know that there is an impact beyond creating jobs, creating wealth, that the solution brings. And assistive technologies are one of those areas that are underserved… not just on the continent, but in the world,” she added.
“[Savatia] has a real solution that works and that can really transform lives.”
Source: edition.cnn.com