PP1315 - Audiovisual Speech Recognition and Training for Adults with Age-Related Hearing Loss

Abstract:
Speech is perceived by the eyes as well as the ears. However, there are large individual differences in adults' recognition of audiovisual speech in noise. With ARHL, being able to make the most of audiovisual speech can substantially ameliorate the effects of noisy and distorted audio. We carried out a study with groups of older adults with ARHL who trained with audiovisual speech in noise (SPIN) or visual-only (VO). AV trainees significantly improved on auditory-only SPIN and VO speech but not AV SPIN; and VO trainees improved on VO speech and AV SPIN. These contrasting results will be reported

Summary:
Rationale/

Purpose:
Hearing aids can improve signal-to-noise ratio (SNR) by approximately 2 to 5 dB. Audiovisual (AV) speech in noise also allows listeners to recognize speech at lower SNRs, with the benefit ranging from approximately 4 to 11 dB. However, the benefits of AV speech differ substantially across individuals, and training to improve recognition of AV SPIN has not received much attention from the research community. We carried out a study to determine whether training with VO speech or AV SPIN results in generalization to untrained AV SPIN and/or to unisensory VO speech or auditory-only (AO) SPIN. We hypothesized that VO training would be more effective for improving AV SPIN, because lipreading ability is associated with the extent of AV SPIN benefits. 

Methods:
In a randomized controlled study, adults with ARHL (mild to moderately severe) (N = 54, ages 39 to 86) were assigned to 10 days of lipreading (VO) or AV SPIN training, or to be in an untrained control group. Training was open-set identification of the words in sentences with feedback. The feedback was printed words that were identified correctly and only the consonants in words that were misperceived. Everyone was tested on untrained AO SPIN, VO, and AV SPIN sentences before and after training. The test sentences were spoken by a talker and a talker not used in training. None of the testing materials were ever re-used. The results were analyzed using mixed effects logistic regression.

Results &

Conclusions:

Results:
In comparisons of results from before and after training versus test-retest results with untrained controls, (1) both VO and AV SPIN training resulted in significant (p < .01) improvements in lipreading, (2) VO training resulted in significant (p < .001) improvements in AV SPIN recognition, and (3) only the AV SPIN training resulted in significant (p  < .05) AO SPIN learning. Direct comparisons of the VO and AV SPIN groups verified that AV training was more beneficial to VO and AO SPIN learning, but VO training was more beneficial to AV SPIN learning.

The average effect size changes from pre- to posttraining were:

Lipreading:
- Control participants improved 3 percentage points (p.pts.)
- AV SPIN trainees improved 11 p.pts.
- VO trainees improved 7.5 p.pts.

Audiovisual speech in noise:
- AV SPIN improved 6.5 p.pts.
- VO participants improved 11.8 p.pts.

Listening in noise: 
- Controls improved 4.2 p.pts.
- AV SPIN improved 8.1 p.pts.
- VO participants improved 5 p.pts.

Conclusions:
Learning differed across the two different training modalities. How might these different results be explained? The results suggest that VO training supported learning of prelexical VO information (i.e., phoneme and phonetic cues), which combines with audio during AV recognition. However AV SPIN training may have mainly supported learning the specific training talker. Results in the literature suggest that exposure to AV SPIN produced by a specific talker can improve recognition of the same talker's VO spin or AO SPIN. Thus, VO training with feedback may result in learning to extract more information from visual speech, while AV SPIN training may support learning a specific talker.

Research supported by NIH/NIDCD.

Learning Objectives:

• Upon completion, participants will be able to explain the importance of audiovisual speech for recognizing speech in noise.
• Participants will be able to relate the concept of perceptual training modality to the concept of cross-modal generalization.
• Participants will be able to explain differences in performance obtained with auditory-only speech in noise, audiovisual speech in noise, and lipreading.