视听一致性对声音响度变化判断影响-influence of audio-visual consistency on judgment of sound loudness change.docxVIP

上海交通大学硕士学位论文摘要 上海交通大学硕士学位论文 摘要 万方数据 万方数据 相较于以往的研究，本课题的贡献及创新性包括： （1）首先，以往的视听交互的研究局限于对持续时间、空间位置等属性 的判断，而本课题研究的范围扩展到了听觉的响度域，并且研究的是视觉和 听觉变化的认知过程； （2）其次，以往的视听交互的研究中，视听刺激大多只同时呈递一次， 而本实验模式中采用先后呈递的两个视听刺激（每个视听刺激中视觉和听觉 刺激同时呈递）作为一个刺激组，两个视听刺激间存在影响，对两者的综合 的理解才能得出对任务的判断。同时，我们选取的刺激模式不仅能够通过行 为学结果反映不同视觉刺激对听觉响度判断的影响，还通过电生理学结果确 定认知过程。更重要的是，如此多的刺激模式涵盖了更多的内容，排除了以 往实验中的特定因素，使实验更具有普遍性。 （3）再次，在该课题的研究过程中，结合了行为学、ERP、差异波和溯 源分析等多种认知神经科学的研究手段，行为学结果为神经电生理分析提供 一定的基础，而两者又相互映衬，从不同的侧面反映出比较统一的结果，溯 源分析则体现出了这一视听认知过程中不同脑区的激活情况，并且不同时间 段的脑地形图的变化特征体现了整个视听认知过程中大脑激活的区域分布特 征，为更好地揭示不同脑区在特定视听认知过程中的作用提供了一定的依据。 关键词：认知，视听交互，听觉响度，皮层激活 VII INFLUENCE OF AUDIOVISUAL CONGRUENCY ON THE AUDITORY INTENSITY CHANGE JUDGMENT ABSTRACT Our human beings collect information from multi-sensory modalities when detecting and perceiving objects and events, while there?s great interaction among different uni-sensory information. Audiovisual integration is extremely common in daily life since vision and audition are the most important channels to perception and hence plays an important role in human cognitive activities. Therefore, audiovisual cross-modal interaction has become a hot topic in recent years. After integration in brain, visual and auditory information are processed to form cognition and then reflect the real world around us. Congruent audiovisual stimuli could enhance cognition while incongruent cross-modal information either adjust perception or disturb our understanding of the world, which is the problem of ?congruency?. At present, the research of simple audio-visual interaction mainly focus on mutual perception influences between certain attributes (duration, position, frequency, etc.) of a single auditory stimulus and some specific attributes (duration, size, position, frequency, etc.) of a single visual stimulation. However, few efforts were made to investigate cognition during the perception progress of audiovisual stimulus change. Is there interaction in this perception progress modal? If so, does VIII ?congruenc