seeing emotion with your ears emotional prosody implicitly guides visual attention to faces看到情感与耳朵情感韵律隐式地引导视觉注意力的面孔.pdfVIP

Seeing Emotion with Your Ears: Emotional Prosody Implicitly Guides Visual Attention to Faces Simon Rigoulot1,2*, Marc D. Pell1,2 1 McGill University, Faculty of Medicine, School of Communication Sciences and Disorders, Montreal, Quebec, Canada, 2 McGill Centre for Research on Brain, Language and Music, Montreal, Quebec, Canada Abstract Interpersonal communication involves the processing of multimodal emotional cues, particularly facial expressions (visual modality) and emotional speech prosody (auditory modality) which can interact during information processing. Here, we investigated whether the implicit processing of emotional prosody systematically influences gaze behavior to facial expressions of emotion. We analyzed the eye movements of 31 participants as they scanned a visual array of four emotional faces portraying fear, anger, happiness, and neutrality, while listening to an emotionally-inflected pseudo- utterance (Someone migged the pazing) uttered in a congruent or incongruent tone. Participants heard the emotional utterance during the first 1250 milliseconds of a five-second visual array and then performed an immediate recall decision about the face they had just seen. The frequency and duration of first saccades and of total looks in three temporal windows ([0–1250 ms], [1250–2500 ms], [2500–5000 ms]) were analyzed according to the emotional content of faces and voices. Results showed that participants looked longer and more frequently at faces that matched the prosody in all three time windows (emotion congruency effect), although this effect was often emotion-specific (with greatest effects for fear). Effects of prosody on visual attention to faces persisted over time and could be detected long after the auditory information was no longer