SEEING WITH YOUR TONGUE: Sensory-substitution devices help blind and deaf people, but that’s just the beginning.

http://www.newyorker.com/magazine/2017/05/15/seeing-with-your-tongue?mbid=social_twitter
by Nicola Twilley

According to one scientist, "You don't see with your eyes, you see with your brain."

The climbers at Earth Treks gym, in Golden, Colorado, were warming up: stretching, strapping themselves into harnesses, and chalking their hands as they prepared to scale walls stippled with multicolored plastic holds. Seated off to one side, with a slim gray plastic band wrapped around his brow, Erik Weihenmayer was warming up, too—by reading flash cards. “I see an ‘E’ at the end,” he said, sweeping his head over the top card, from side to side and up and down. “It’s definitely popping—is it ‘please’?” he asked me. It was. Weihenmayer moved triumphantly on to the next card.

Erik Weihenmayer is the only blind person to have climbed Mt. Everest. He was born with juvenile retinoschisis, an inherited condition that caused his retinas to disintegrate completely by his freshman year of high school. Unable to play the ball games at which his father and his brothers excelled, he took to climbing after being introduced to it at a summer camp for the blind. He learned to pat the rock face with his hands or tap it with an ice axe to find his next hold, following the sound of a small bell worn by a guide, who also described the terrain ahead. With this technique, he has summited the tallest peaks on all seven continents.

A decade ago, Weihenmayer began using the BrainPort, a device that enables him to “see” the rock face using his tongue. The BrainPort consists of two parts: the band on his brow supports a tiny video camera; connected to this by a cable is a postage-stamp-size white plastic lollipop, which he holds in his mouth. The camera feed is reduced in resolution to a grid of four hundred gray-scale pixels, transmitted to his tongue via a corresponding grid of four hundred tiny electrodes on the lollipop. Dark pixels provide a strong shock; lighter pixels merely tingle. The resulting vision is a sensation that Weihenmayer describes as “pictures being painted with tiny bubbles.”

Reading the cards before his climb helped Weihenmayer calibrate the intensity of the electrical stimulation and make sure that the camera was pointing where he thought it was pointing. When he was done, he tied himself into his harness and set off up Mad Dog, a difficult route marked by small blue plastic holds set far apart on the wall. Without the BrainPort, Weihenmayer’s climbing style is inelegant but astonishingly fast—a spidery scramble with arms and feet sweeping like windshield wipers across the wall in front of him in order to feel out the next hold. With the device on his tongue, he is much slower, but more deliberate. After each move, he leans away from the wall, surveys the cliff face, and then carefully reaches his hand out into midair, where it hovers for a split second before lunging toward a hold several feet away. “You have to do the hand thing, because it’s hard to know where, exactly, things are in space,” Weihenmayer explained, as I prepared to tackle Cry Baby, a much simpler route. “Once my hand blocks the hold, I know I’m in front of it, and then I just kind of go in there.”

Weihenmayer told me that he wouldn’t take the BrainPort up Everest—relying on fallible electronics in such extreme conditions would be foolhardy. But he has used it on challenging outdoor climbs in Utah and around Colorado, and he loves the way that it restores his lost hand-eye coördination. “I can see the hold, I reach up, and I’m, like, ‘Pow!’ ” he said. “It’s in space, and I just grabbed it in space. It sounds so simple when you have eyes, but that’s a really cool feeling.”,

The BrainPort, which uses the sense of touch as a substitute for sight, is one of a growing number of so-called sensory-substitution devices. Another, the vOICe, turns visual information into sound. Others translate auditory information into tactile sensation for the deaf or use sounds to supply missing haptic information for burn victims and leprosy patients. While these devices were design,d with the goal of restoring lost sensation, in the past decade they have begun to revise our understanding of brain organization and development. The idea that underlies sensory substitution is a radical one: that the brain is capable of processing perceptual information in much the same way, no matter which organ delivers it. As the BrainPort's inventor, the neuroscientist Paul Bach-y-Rita put it, "You don't see with your eyes, you see with your brain."

Bach-y-Rita, who died in 2006, is known as the "father of sensory substitution," although as he liked to point out, both Braille and white canes are essentially sensory-substitution systems, replacing information that is typically visual -- words on a page, objects at a distance -- with tactile sensation. He even argued that writing ought to be the original precursor, because it enabled the previously auditory experience of the spoken word to be presented visually......

~~~~~~~~

NOTE: Since the article is pages long, I thought I'd stop here. If you are interested in the rest, please click on the link at the top of page which will take you to the original.

It's fascinating, isn't it?

I noticed after having yogurt for breakfast that even while eating, I was taken at how diluted it was compared to the same yogurt 20 years ago. Wholeheartedly agree with this concept.

Incredible the blind and deaf will be able to enjoy all that the planet and its inhabitants have to offer.