With only her mouth, this 19-year-old painter toggles a joystick positioned next to her face to move around on a wheelchair.
Due to circumstances surrounding her premature birth, Elaine Eng was born with cerebral palsy — a lifelong, physical disability that affects both movement and posture.
Despite her limitations, Elaine enjoys painting, and lays an array of colours onto an empty canvas using a paintbrush held in her mouth.
At the moment, Elaine receives assistance from volunteers at the Cerebral Palsy Alliance Singapore (CPAS), who help her to move the canvas and dispense paint.
In August 2018, a team led by Dr Ong Lay Teen, a Senior System Engineer at the Government Technology Agency of Singapore (GovTech), visited CPAS to discuss how they could apply new technologies to help those with physical disability.
“We were introduced to Elaine and was told that she is a mouth painter,” recalled Dr Ong. “It all began from then when we saw her passion to paint.”
A Dream Come True For Mouth Painters
Moved by Elaine’s love for painting, the team decided to build an eCanvas, a voice-controlled device that would provide greater independence for her artistic endeavours.
The eCanvas consists of a motorised aluminium frame that clamps onto a drawing canvas and moves the canvas within the reach of the painter.
A microphone connected to the eCanvas receives voice commands of the painter, either to move the canvas or dispense paint on a palette. A mini-computer installed with a pre-trained neural network engine processes the voice commands.
“At the beginning, the idea was to have a canvas that moved up and down,” said Quek Yang Boon, Director of Smart Nation Systems and Solutions, Sensors and Internet of Things at GovTech.
But the project soon evolved into something bigger, with the paint dispenser as an example of the team’s ingenuity.
Improvised from simple, off-the-shelf items, the paint dispenser consists of a number of soap dispensers that had been repurposed to release paint onto a turntable, which acts as the palette.
“Each person involved [in the eCanvas project] had a different skillset that happened to blend very well together,” said Quek.
Parts of the eCanvas, such as the paintbrush holder and circuit board, were specifically designed or sourced for by the team. “Every single bolt and nut that’s on there was screwed on by us. I believe that is what true engineering is about,” he added.
Perfecting The eCanvas’ Design
Even features that were originally planned, such as the voice command system, had to be revised several times before they could perform well enough.
“We imagined that the solution would be this or that,” Dr Ong shared, “but when we tested the eCanvas on site, we realised that we had to change the design for it to be more applicable to Elaine.”
Instead of a generic system that could recognise many words, the team built a custom system — powered by a convolutional neural network — to recognise only a few keywords for better performance.
To further increase the system’s accuracy, Elaine’s voice was added into the mix when training the device. The voice control system interface was designed to provide a seamless user experience for Elaine.
She did not have to navigate through levels of voice commands like in a phone menu, as the options were parallelised onto a single layer. This meant that Elaine would only have to say a specific command for a desired action to occur.
Enabling One For Many
The eCanvas project not only demonstrates how technology can be used to improve lives, but also showcases the engineering and development capabilities within GovTech.
From hardware to software, the Sensors and Internet of Things (SIOT) team has the expertise to potentially make a difference in the assistive technologies space, said Quek.
For instance, the ideas and solutions put forth to cater to those with physical disabilities can be adapted to serve other segments of the population as well, such as the elderly, which will comprise a quarter of Singapore’s population by the year 2030.
All this is aligned with GovTech’s focus on inclusiveness, whereby no individual is left behind as Singapore digitalises and strives to become a smart nation.
“As part of the government, we tend to always look at the big picture and to take on a lot of big projects,” said Quek. “What I think the eCanvas shows is that we can also look at the value that technology can create for smaller segments of society who may be overlooked.”
- This article first appeared on GovTech. You can also get byte-sized (geddit?) tech stories by GovTech here.
Featured Image Credit: GovTech