EmailEmail
PrintPrint
Text with an edge
CMU researchers developing an entry system for PDAs that could help people with impaired motor skills
Monday, November 29, 2004

As computers grow smaller, the problem of text entry just gets bigger.

Robin Rombach, Post-Gazette
The developers: Jacob Wobbrock, a fourth-year gradute student at Carnegie Mellon University, and computer science professor Brad Myers.
Click photo for larger image.

Graphic: EdgeWrite alphabet rendition, in .pdf format

Handheld devices, such as cell phones and personal digital assistants, or PDAs, can easily fit in a pocket. But without a traditional keyboard, how do you type a short note or e-mail message?

Many users find the available options less than satisfactory, but the problem is particularly troubling for people with motor impairments, such as Parkinson's disease or cerebral palsy, who have difficulty moving a stylus over the a PDA screen with any degree of accuracy.

A new system being developed at Carnegie Mellon University's Human-Computer Interaction Institute uses the edges of a square overlay to guide the stylus over the touchscreen, thus making it easier for impaired users to input text.

Called EdgeWrite, the system also could help any user of a PDA who is trying to write while walking or riding in a vehicle -- people with so-called "situational impairments." And Jacob Wobbrock, the graduate student who is developing EdgeWrite with computer science professor Brad Myers, said it could be adapted to a number of controllers, such as joysticks, and for even more compact devices, such as MP3 players and wristwatch computers.

"New devices come out all of the time," Wobbrock said. "It'd be nice if we could have one [text entry] technique and have it work on a number of devices."

"My impression is people are not particularly happy with what's available," said Michael Fleetwood, a graduate student at the Computer-Human Interaction Laboratory at Rice University in Houston. "A lot of people are working on it."

Some text entry techniques have obvious drawbacks. Collapsible keyboards are an extra piece of equipment to carry around. Voice recognition isn't reliable enough to let users simply speak into a device. And punching in letters using a cell phone's numeric keypad is cumbersome.

The most successful approaches are in two main categories -- "virtual keyboards," which display alphabet keys on a touch screen, and electronic shorthand, such as EdgeWrite or the Graffiti system used in the popular Palm Pilot.

Neither of these systems match the speed of a traditional keyboard, though virtual keyboards are theoretically the faster of the two approaches, Fleetwood said.

The QWERTY arrangement of standard keyboards, developed in 1868 to minimize jamming of typewriter keys by two-handed typists, is widely used on virtual keyboards, but isn't well-suited for punching with a single stylus. Shumin Zhai, a researcher at IBM Almaden Research Center, has developed an optimized virtual keyboard called ATOMIK -- with the spacebar in the center, surrounded by the most commonly used letters, E, N, I, S, O, T -- that is capable of about 40 words per minute. That's a bit less than a slow typist on a standard keyboard, Fleetwood said.

Users must look at the virtual keyboard to use it -- there's no such thing as touch typing -- and must have a steady hand. And it takes a lot of practice to increase speed.

Graffiti and other shorthand systems are more like writing, are fairly easy to learn and don't require the user to keep an eye on the screen all the time. But Fleetwood said they generally aren't much faster than printing letters on a piece of paper, about 20 words per minute.

Wobbrock said EdgeWrite is about as fast and as easy to learn as Graffiti, but he acknowledges that more needs to be done to increase the speed of entry. "I would say we're halfway through the technology development," he added.

No one would want to use one of these handheld devices to write a book anyway, Myers said.

Outgrowth of another project

That handhelds could be particularly useful for people with disabilities was brought home to Myers several years ago. At that time, an undergraduate project produced a method for using the PDA touchscreen to control the cursor on a remote computer.

Robin Rombach, Post-Gazette
Part of the EdgeWrite system is a plastic panel with a square cutout that is placed over the PDA screen, creating the edges for the user's stylus to slide against.
Click photo for larger image.
"We hadn't intended it to be an application for people with handicaps," Myers said. Rather, the idea was people in a meeting could use their PDAs to take turns operating a common computer. It proved fairly successful and the software was made available for download from the Web.

Myers subsequently got an email from a man who said the software, Remote Commander, helped his daughter, who had muscular dystrophy. She had little strength in her hands and she found the small touchscreen was easier for her to control than a standard keyboard or mouse.

That got Myers and his students thinking about other adaptive uses of PDAs. People with jittery hands, such as those with Parkinson's or cerebral palsy, had trouble inputting text with PDAs (as well as with conventional keyboards). The small keys of virtual keyboards require hand dexterity and shorthand techniques such as Graffiti require smooth, accurate movements, such as drawing curves and straight lines.

Neither Myers nor Wobbrock recall who got the idea to place a piece of plastic with a square cut out of it over the PDA screen. But it worked: even someone with a jittery hand can run a stylus along the edges of the square. So they just had to develop an alphabet that could be formed by straight lines, either along the edges or diagonally across the square.

"I don't think most people believed you could develop an intuitive alphabet in a square," Myers said, noting most alphabets are "loopy." But they found the strokes could approximate those used in printing a letter ---- for instance, a "b" is formed by stroking down along the left side of the cutout and then right to the lower right corner and back to the lower left corner.

As a result, the strokes surprisingly end up feeling a lot like natural printing, Wobbrock said.

Though users form the letters with "unistrokes" ---- lifting the stylus only after completing a letter ---- the device doesn't recognize the letters based on the shape of the strokes, but on the sequence of corners that are hit when forming the letters. So if the stylus doesn't stay precisely along an edge, or a diagonal stroke looks more like a bell curve than a straight line, the correct letter is entered as long as the user hits the corners in the right sequence.

"It seems like a very good solution," Fleetwood said.

What's more, because the technique relies not on pattern recognition but the sequence of corners, or points, EdgeWrite can be used with other devices, such as joysticks used to control videogames ---- or power wheelchairs.

"So many adaptive aids require learning a whole new gadget," said Janis Thoma-Negley, a curriculum specialist with United Cerebral Palsy of Pittsburgh. A wheelchair user who already is accustomed to using the chair's joystick thus would welcome using the same joystick to operate a computer.

Wobbrock and his colleagues were able to adapt a commercial wheelchair joystick to this double duty and, in tests last spring, it impressed a couple of United Cerebral Palsy clients.

"Boy, they hope they do put it out on the market," Thoma-Negley said of the two women who tested the system.

From edges to bumps

Wobbrock is continuing to develop the system. For instance, one version dispenses with the overlay and the stylus and uses just four raised Braille-like bumps on a finger touchpad. Users can enter a letter simply by rubbing a fingertip over the bumps in the right sequence.

Such a touchpad could be quite small, with the bumps only a quarter of an inch apart, he said. That's small enough to place on the front or the back of a cell phone. It would fit neatly into the center of an iPod scroll ring, allowing searches of songs by letter. Or it could fit on a wristwatch.

How widely EdgeWrite might be used is anyone's guess. The researchers keep in touch with Microsoft, which funds some of the work, though Myers said Microsoft officials have given no indication they plan to use it anytime soon. Some makers of wheelchair joysticks have expressed interest.

In the meantime, those who want to try EdgeWrite can download the software for free at www-2.cs.cmu.edu/~edgewrite/. Remote Commander and other PDA software also is available at www-2.cs.cmu.edu/~pebbles/.

First published on November 29, 2004 at 12:00 am
Post-Gazette science editor Byron Spice can be reached at bspice@post-gazette.com or 412-263-1578.
EmailEmail
PrintPrint