RP304 - Lipreading and Audiovisual Speech-in-Noise Training with Sentence Stimuli for Older Adults with Hearing Loss: Learning and Generalization Result when High Contingency Feedback is Provided

Abstract:
We have been investigating how feedback can be designed to achieve learning with generalization during speech recognition training (with sentence stimuli) for adults with age-related hearing loss (ARHL). In a successful randomized controlled trial of lipreading and audiovisual speech-in-noise (AV SPIN) training, different groups received feedback that differed in whether, or how, it presented information about their perceptual errors. Learning with generalization to untrained stimuli was obtained when feedback was given for both perceptual errors and words correct. Whole-sentence feedback produced significantly worse recognition. Learning patterns differed significantly between lipreading and AV SPIN training, suggesting the use of both approaches.

Summary:

Purpose:
The holy grail of speech recognition training studies has long been to show generalization to untrained speech materials, different talkers, and varied speech recognition tasks. Based on perceptual learning and cognitive neuroscience research and theory, we hypothesized that the information in feedback given during training controls learning. We also hypothesized that lipreading training will generalize to untrained audiovisual (AV) speech-in-noise (SPIN) speech recognition.

Methods:
In a randomized controlled study, older adults (OA) with age-related hearing loss (ARHL) (mild to moderately severe) (N = 87, ages 39 to 85) were assigned to 10 days of lipreading (visual-only, VO) or AV SPIN training or to be untrained controls. Participants were assigned to different feedback types based on the level of contingency between the participant's response and the content of the feedback. Feedback contingency is considered to be high when there is a predictable and learnable relationship between performance and feedback. Traditional whole sentence feedback (SentFB) is low contingency as it does not depend at all on the response. Our word feedback (WordFB) was a medium contingency because it displayed correct words on green and misperceived words on amber. Our highest contingency type was Consonant feedback (ConsFB) because it displayed words correct on green and only the consonants in misperceived words, supporting attention to prelexical perceptual errors. Real-time sequence alignment technology was used to score responses and deliver feedback during remote training sessions. Remote, pre- and posttraining tests included AO SPIN, AV SPIN, or VO untrained isolated sentences and words spoken by the training talker and by unfamiliar talkers. The training stimuli were VO speech or AV SPIN. 

Results:
Training with VO or with AV SPIN sentences and ConsFB resulted in learning with statistically significant generalization to untrained stimuli and talkers. The most successful feedback for learning with generalization was ConsFB when compared to test-retest by untrained controls. Most participants improved, average improvements were approximately 10 to 11 %age points (p.pts), and the top quartile of participants mostly improved more than 12.5p.pts. across test modalities. When the ConsFB trainees who trained with VO speech were compared with those who trained with AV SPIN, there were significant interactions of training modality (AV SPIN, VO) with test modality (AV SPIN, AO SPIN, VO). VO training was significantly more effective for generalization to AV SPIN stimuli. However. AV SPIN training was significantly more effective for AO SPIN and VO stimuli. Notably, SentFB with VO speech resulted in significantly worse scores at posttraining relative to untrained controls. WordFB mostly benefited AV SPIN sentence and word recognition. 

Conclusions:
The information that is delivered during feedback has a powerful effect on what is learned. So too does the training stimulus modality. Learning was achieved by adults as old as 85 who trained with their computers at home. Compliance was high with only a few dropping out of the study. Current ConsFB training with AO SPIN stimuli is also showing indications of being effective. ConsFB during training with ecologically valid stimuli is a feasible, effective method for achieving significant learning.