Many individuals with severe motor impairments can communicate using computer interfaces where they select letters or words in an onscreen grid. They need to activate a single switch, often by pressing a button, releasing a puff of air, or blinking.
However, these systems are very rigid and highlight each option one at a time, making them frustratingly slow for some users. Plus, they are not suitable for tasks where options can’t be arranged in a grid, like drawing, browsing the web, or gaming.
MIT scientists have introduced a new system called Nomon, in which individual selection indicators are placed beside each option on a computer screen. It includes probabilistic reasoning to learn how users make selections and then adjust the interface to improve speed and accuracy.
When scientists tested their system, the participants were able to type faster. They also performed better on a picture selection task, demonstrating how Nomon could be used for more than typing.
Scientists placed a small analog clock within the Nomon interface. The user looks at one option and clicks their switch when that clock’s hand passes a red “noon” line. The system changes the phases of the clocks after each click to separate the most probable next targets. The user repeatedly clicks until their target is selected.
When used as a keyboard, Nomon’s machine-learning algorithms try to guess the next word based on previous words and each new letter as the user makes selections.
Using a newly developed webcam-based switch, the team collected more representative data. The switch was more challenging to use than simply clicking a key. The non-switch users had to lean their bodies to one side of the screen and then back to the other side to register a click.
Senior author Tamara Broderick, an associate professor in the MIT Department of Electrical Engineering and Computer Science (EECS), said, “And they have to do this precisely the right time, so it slows them down. We did some empirical studies which showed that they were much closer to the response times of motor-impaired individuals.”
They ran a 10-session user study with 13 non-switch participants and one single-switch user with an advanced form of spinal muscular dystrophy. In the first nine sessions, participants used Nomon and a row-column scanning interface for 20 minutes each to perform text entries, and in the 10th session, they used the two systems for a picture selection task.
Nomon allows non-switch users to type 15 percent faster, whereas the motor-impaired user typed even quicker than the non-switch users. While typing unknown words, the users were 20 percent faster overall and made half as many errors. The participants completed the picture selection task 36 percent fast in the final session.
Broderick says, “Nomon is much more forgiving than row-column scanning. With row-column scanning, even if you are just slightly off, you’ve chosen B instead of A, and that’s an error.”
“Nomon incorporates everything it knows about where a user is likely to click to make the process faster, easier, and less error-prone. For instance, if the user selects “Q,” Nomon will make it as easy as possible for the user to select “U” next.”
“It also adapts to noisiness. If a user’s click is often off the mark, the system requires extra clicks to ensure accuracy.”
The research is presented at the ACM Conference on Human Factors in Computing Systems.