多模态 Collin.pptVIP

Multimodality and its Educational Application 周东方 What to share with you? What’s multimodality? The origin, history, latest development Educational applications Communicative modes Visual modality Auditory modality Tactile modality Olfactory modality Gustatory modality The origin of multimodality Time: 1920s Technical term Psychology of perception (直觉心理学） Usage: denote the effect different sensory perceptions have on each other Example: McGurk effect McGurk and MacDonald found this effect，which proves to be universally appliable。 Linguist and discourse analyst Broaden it to denote the integrated use of different communicative resources, such as language, image, sound and music in multimodal texts and communicative events. Texts and communicative events Communication is multimodal Spoken language cannot be adequately understood without taking non-verbal communication into account Many forms of contemporary written language cannot be adequately understood unless we look not just language but also at images, layout, typography and color. In the past 20ys or so A field of study Investigate the common and distinct properties of different modes in the multimodal mix and the way they integrate in multimodal texts and communicative events Development 20th century Several schools of linguistics engaged with communicative modes other than language. 1930s-1940s Prague School 1960s Paris School Roughly the same time, American linguists 1990s The fourth school Further approach Prague School 1930s-1940s Linguistics→ visual arts non-verbal aspects of theatre It includes studies of folklore and collaboration with avant-grade artists Pairs school structuralist semiotics 1960s Use concepts and methods from linguistics to understand communicative modes other than language Inspired by Roland Barthes Focus on analysis of popular culture and the mass media, rather than on folklore and avant-grade art. American linguists 1960s-1970s Multimodal analysis of spoken language and non-verba