Opening new portals for the blind

Bryn Nelson Columnist

For blind students, surfing the Web or taking detailed notes in physics and calculus classes can require cumbersome and expensive learning aids. Two new technologies, however, promise to greatly expand access to text and graphics for the blind community without breaking the bank.

The Pulse Smartpen, which debuts in retail stores this month, records and synchronizes  nearly everything a student hears and writes. And WebAnywhere, a free Internet-based service released last month, works as a screen reader by converting Web site text to electronic speech.

Beyond meeting an immediate need in the classroom or office, some researchers believe the products could help dramatically expand career options for the blind by opening new doors to information.

“Many of the solutions that are provided to this community are pretty expensive because the community is small,” said Andy Van Schaack, a lecturer in Vanderbilt University’s Peabody College of Education and Human Development in Nashville, Tenn. But by focusing on a consumer product marketed to the general public, he and colleague Joshua Miele, an associate scientist at the Smith-Kettlewell Eye Research Institute in San Francisco, have found an economical way to close the accessibility gap.

The audio-tactile graphic system resulting from their research centers on the Pulse Smartpen by Oakland start-up Livescribe, for which Van Schaack serves as a senior science advisor.

The pen works just like a regular one if it’s not turned on, he said “But when you turn it on, it’s now actually a computer.”

The smartpen recognizes handwriting through a small infrared camera at its tip that snaps 72 pictures per second, focusing on millions of tiny dots printed on every page of a special notebook accompanying the pen.

“It actually takes a picture of the page and what’s printed onto each page, which is a very fine, almost invisible dot pattern that’s almost like a barcode pattern,” Van Schaack said. The notebook’s unique microdot patterns can help the pen pinpoint the location of a word on the top of page 11, say, instead of the bottom of page 12.

When a note-taker taps the image of a record button on a control strip along the bottom of every notebook page, twin microphones on the pen synchronize the notes to whatever is being said at the time. Tapping on the notes later — a calculus equation, for example — recalls the relevant digital audio recording, such as the teacher’s explanation for each symbol in the equation.

The smartpen has enough memory to capture up to 100 hours of audio or 30,000 pages of notes. Both audio and visual elements can be uploaded to a computer via a USB cradle that also recharges the pen, and the pen can play back recorded audio either through its tiny speaker or through a headset attached to an audio jack.

Putting a finger on problem graphics
Miele said he immediately recognized the pen’s potential for helping blind people, and in particular, blind students taking science, technology, engineering or math classes.

“Spatial information like charts and maps are heavily used in technical fields,” he said. “Blind people are historically underrepresented in those fields because it’s hard to represent all of the graphical information for blind students.”

As a blind scientist himself, he said, “it’s important to push those numbers in the right direction.”

Braille, roughly equivalent to 24-point type, has traditionally permitted only a few words on complex figures and tables due to size constraints. Furthermore, Miele said only about 10 percent of blind people are literate in Braille. With its ability to produce audio labels, the pen system could be used not only by people who don’t yet know Braille, but could also help them to learn it. “If you don’t know what a character is, you could tap it with the pen and it could tell you,” he said.

The blind can now publish professional-quality, pen-enabled materials such as maps, periodic tables, economic tables, bar charts and organizational charts on a Braille embosser — or printer — all embedded with audio tags based on the printed microdot patterns instead of the much larger dots required for Braille, Miele said.

“You can annotate stuff to your heart’s content,” he said. “You can layer stuff. Tap it once, and it tells you what it is. Tap it twice, and it may give you supplemental information. You can keep on layering information that would be absolutely impossible if you were to do it in Braille.”

Aided by a $300,000 grant from the National Science Foundation, the researchers are also using the smartpen in conjunction with what’s known as a Sewell Raised Line Drawing Kit. “Think of it as a clipboard that has a rubberized surface — your thumbnail would leave an impression,” Van Schaack said.

A clear plastic transparency over the clipboard allows users to draw an image that will leave a physical impression. “The [teaching assistant] can say something like, ‘The teacher just drew a triangle,’ while drawing it on the clipboard, and you can feel it,” he said. “So it’s a tactile figure.”

The smartpen, used in conjunction with the Sewell kit, records not only the drawing’s location but also what was said when it was drawn. Later, a student can tap on the raised triangle and hear the associated classroom commentary as a site-specific auditory label.

The researchers will eventually bring in students on an individual basis and gauge how well they can use the combination of tools as learning aids. Later on, Van Schaack hopes to train students and teaching assistants and let them use the system in real classroom settings, “so we can figure out how what we’ve learned in the lab translates to the real world.” 

A smartpen with one gigabyte of memory sells for $149, while a pen with two gigabytes goes for $199. A four-pack of 100-page notebooks sells for $19.95, though Van Schaack said users eventually will be able to print out their own notebook paper.

MORE FROM FRONTIERS

Tourist or terrorist? Computer knows

Tourist or terrorist? Computer knows Climbing no obstacle for snakebots The next 'American Idol'? Ask your computer How a deadly avalanche can be triggered Green alternative to plastics: liquid wood Getting lost for better architecture Turning air into drinking water Revving up the race for fuel efficiency How to be a disagreeable (but likable) robot Can concept of clean coal be salvaged?   Add Frontiers headlines to your news reader: