New Finding Has Implications for Scientists Designing New Mobile Audio Interfaces

06/24/03    Portland, Ore.

Researchers at OHSU's OGI School of Science & Engineering are working to make computers more human-friendly

Oregon researchers have discovered that when people converse with text-to-speech (TTS) computer systems, they substantially change their speech to sound like the computer -- what's known as speech convergence. In addition, people will readapt their voice to each new text-to-speech voice they hear. The findings are important for scientists who are designing new mobile audio interfaces, according to Sharon Oviatt, Ph.D., an experimental psychologist who led the study at Oregon Health & Science University's OGI School of Science & Engineering.

"Human speech is always a variable," said Oviatt, who co-directs the Center for Human-Computer Communication in the Department of Computer Science and Engineering at OGI. "We discovered that people subconsciously tailor their voice to the computer's voice. So, for example, if the computer voice is talking more quietly, the person interacting with the computer will talk more quietly. This is a major new source of variability in users' spoken language to computers, which is opening up new scientific directions for mobile audio interface design."

As part of her study, funded by the National Science Foundation, Oviatt and her graduate students asked students aged 7 to 10 to use a special handheld computer to talk with a variety of digital marine animals as they learned marine biology. Students in the "I SEE!" Program (Immersive Science Education for Elementary Kids) used the handheld computer to speak directly to or write with a pen onto a screen questions for animated software characters. The marine animals answered each question using text-to-speech output, along with animated movement.

Four different TTS voices were used to determine whether the children accommodated their own voices (amplitude, duration, pitch, etc.) to more closely match the TTS output from the computer characters, or what is called speech convergence. Speech convergence is known to occur in human-human communication, but this is the first time it has been studied as part of human-computer interface design.

Oviatt and her team are now modeling the results from these and future experiments so they'll be able to reliably predict how speech will vary under certain circumstances. Such quantitative modeling, they say, will lead to a new science of audio interface design, which will be important for developing new mobile systems that are capable of processing users' speech reliably in natural settings.

"Ideally, we want to develop high-functioning applications and interface designs for specialized populations," said Oviatt. "This is a tall order for computers right now."

Most people now have one way to communicate with their computer: through a keyboard. Oviatt and collaborator husband, Phil Cohen, Ph.D., are pioneers in the research and design of multimodal interfaces -- that is, computer systems that enable people to communicate with them using more than one modality. The pair have made breakthroughs that empower people to interact with computers via speech, pen, touch and gesture.

"Currently humans adapt to the limitations of computers," said Oviatt. "But future computers need to be more adaptive to us. They should be smaller and embedded so we don't have to carry them. They should be able to combine pen and speech input to express themselves naturally and efficiently, and with information tailored to an individual's communication patterns, usage patterns, and physical and cognitive needs."

The Center for Human-Computer Communication at the OGI School of Science & Engineering has four full-time faculty, seven graduate students, and additional programming staff. For more information, including video and audio clips, go to and click on "Research Projects and System Description" and "I SEE!"

The OGI School of Science & Engineering (formerly the Oregon Graduate Institute of Science & Technology) became one of four schools of the Oregon Health & Science University in 2001. The Department of Computer Science and Engineering at the OGI School is ranked 15th nationally for its federally funded computer research.