学习者口语语料自动词性赋码信度研究.docVIP

学习者口语语料自动词性赋码信度研究 　　摘 要：词性赋码是对语料库进行句法分析的必要前提，可极大地拓宽语料库研究范围。而现有的英语自动词性赋码器多为本族语的书面语设计，能否准确地为学习者口语语料进行自动赋码尚未见到研究报告。本研究采用Brill和CLAWS7两种词性赋码器分别为高低分组学习者口语语料进行自动赋码，统计赋码准确率，并比较两种赋码器对学习者口语特征的处理能力，目的在于考察基于规则的词性赋码器和基于概率的词性赋码器对学习者口语语料进行自动词性赋码的适用度，以期为中介语口语的句法特点研究提供依据。研究结果与梁茂成对中国学生书面语赋码准确率的考察结果基本一致，基于概率的词性赋码器更适用于为学习者口语语料进行自动赋码，赋码准确率较高且性能稳定，其赋码准确率受学习者口语语言水平影响不大，而基于规则的词性赋码器受学习者语言水平和口语特征（停顿、重复、缺乏语法性等）的影响较大。本研究为学习者口语句法特点研究提供重要依据，对误码编辑（tag editing）也有一定意义。 　　关键词：词性赋码；中介语口语；语言水平；准确率 　　中图分类号：H319.9文献标识码：A文章编号：1000-5544(2007)04-0047-05 　　Abstract： POS tagging is an essential prerequisite for further syntactic analyses and important in widening the research scope. However, corpus-based studies on syntactic aspects of second language acquisition have been rare due to the dubious reliability of POS taggers programmed to tag native speakers written data. In this study, the Brill POS tagger and the CLAWS7 POS tagger have been used to assign POS tags to the same set of transcribed spoken data produced by a high-proficiency group and a low-proficiency group of EFL learners. The POS tags have then been manually examined, and the success rates of the taggerscompared. Findings of the study support Liangs conclusion that the probability-based tagger is more viable and less affected by the proficiency level of the learners. Besides, the taggers are also examined for their ability to handle spoken features in the texts. It is revealed that pauses, repetitions and sentence fragments in learners transcribed spoken data seriously affect the performance of the rule-based tagger. The study sheds important light on the feasibility of corpus-based studies on syntactic aspects of spoken interlanguage, and offers some implications for tag editing. 　　Key words: POS tagging; spoken interlanguage; proficiency level; accuracy 　　 　　1．引言 　　 　　20世纪60年代 Kucera 和 Francis 建立了世界上第一个计算机语料库―BROWN 语料库，标志着基于计算机语料库研究的开始。经过40年的蓬勃发展，语料库方法已被广泛地应用到自然语言处理、词典编纂、教材开发、机器翻译、语言教学、数据驱动语言学习等领域。由于语料库数据提取的便捷性和准确性，研究方法的规范性和科学性，研究结