cross-modal prediction in speech perception跨通道在言语知觉预测.pdfVIP

Cross-Modal Prediction in Speech Perception ´ ´ 1 ` 2 3 1,4 Carolina Sanchez-Garcıa , Agnes Alsius , James T. Enns , Salvador Soto-Faraco * ´ 1 Departament de Tecnologies de la Informacio i les Comunicacions, Universitat Pompeu Fabra, Barcelona, Spain, 2 Department of Psychology, Queen’s University, ´ Kingston, Canada, 3 Department of Psychology, University of British Columbia, Vancouver, Canada, 4 Institucio Catalana de Recerca i Estudis Avanc¸ats (ICREA), Barcelona, Spain Abstract Speech perception often benefits from vision of the speaker’s lip movements when they are available. One potential mechanism underlying this reported gain in perception arising from audio-visual integration is on-line prediction. In this study we address whether the preceding speech context in a single modality can improve audiovisual processing and whether this improvement is based on on-line information-transfer across sensory modalities. In the experiments presented here, during each trial, a speech fragment (context) presented in a single sensory modality (voice or lips) was immediately continued by an audiovisual target fragment. Participants made speeded judgments about whether voice and lips were in agreement in the target fragment. The leading single sensory context and the subsequent audiovisual target fragment could be continuous in either one modality only, both (context in one modality continues into both modalities in the target fragment)