Voice pitch as an aid to lipreading

This work aims to isolate relatively simple patterns of stimulation of the cochlea that, although not intelligible in themselves, will aid lipreading and has implications for other sensory substitution techniques, the design of special purpose hearing aids and current theories of speech perception.

Abstract

The totally deafened adult, unable to make use of a hearing aid, has no alternative to lipreading for everyday communication. Lipreading, however, is no substitute for hearing speech. Many lipreaders have great difficulty in ideal conditions and even the best lipreaders find the task demanding and tiring. Prosthetic attempts to substitute for lost hearing have centred on three distinct types of intervention, visual, tactile, and electrocochlear. As none of these is likely to yield a good understanding of a speech independent of lipreading in the near future, we have attempted to isolate relatively simple patterns of stimulation that, although not intelligible in themselves, well aid lipreading. From this point of view, the fundamental frequency or 'pitch' of the voice is the most important pattern element because if provides both segmental and suprasegmental information and is practically invisible. It thus complements the visual information already available on the face. As we show here, with the voice pitch presented acoustically, normal listeners can lipread a speaker reading continuous text at up to two and a half times the rate possible on the basis of lipreading alone. The pitch signal by itself, of course, is completely unintelligible. Although our work is primarily concerned with methods of electrical stimulation of the cochlea, it has implications for other sensory substitution techniques, the design of special purpose hearing aids and current theories of speech perception.