中文分词与词性标注联合模型综述.docVIP

中文分词与词性标注联合模型综述 　　收稿日期：2014－05－09 　　基金项目：国家自然科学基金。 　　作者简介：赵芳芳(1990-),女,河南许昌人,硕士研究生,主要研究方向: 自然语言处理; 　　蒋志鹏(1985-),男,黑龙江七台河人,博士研究生,主要研究方向: 自然语言处理; 　　关毅(1970-),男,黑龙江宁安人,博士,教授,博士生导师,主要研究方向: 用户健康信息学、网络挖掘、自然语言处理等。 　　 　　 　　摘要：中文分词和词性标注任务作为中文自然语言处理的初始步骤，已经得到广泛的研究。由于中文句子缺乏词边界，所以中文词性标注往往采用管道模式完成：首先对句子进行分词，然后使用分词阶段的结果进行词性标注。然而管道模式中，分词阶段的错误会传递到词性标注阶段，从而降低词性标注效果。近些年来，中文词性标注方面的研究集中在联合模型。联合模型同时完成句子的分词和词性标注任务，不但可以改善错误传递的问题，并且可以通过使用词性标注信息提高分词精度。联合模型分为基于字模型、基于词模型及混合模型。本文对联合模型的分类、训练算法及训练过程中的问题进行详细的阐述和讨论。 　　关键词：中文分词; 中文词性标注; 联合模型 　　中图分类号：TP391文献标识码：A文章编号：2095-2163（2014）03-0077-04 　　The Review on the Joint Model of Chinese Word Segmentation 　　and Part-of-speech Tagging 　　ZHAO Fangfang, JIANG Zhipeng, GUAN Yi 　　(School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China) 　　Abstract：Chinese word segmentation and part-of-speech (POS) tagging task as an initial step for Chinese natural language processing, has been widely studied. Due to the lack of Chinese sentences word boundary, the Chinese POS tagging task is often completed with the pipeline approach: firstly, perform Chinese word segmentation, and then use the results of the prior stage to tag the Chinese sentence. However, in the pipeline approach, word segmentation phase errors will be passed to the POS tagging stage, thereby reducing the accuracy of POS tagging. In recent years, the research on Chinese POS tagging focused on the joint model. The joint model perform both word segmentation and POS tagging in a combined single step simultaneously, through which the error propagation can be avoided and the accuracy of word segmentation can be improved by utilizing POS information. There are character-based methods, word-based methods, and hybrid methods. In this paper, the three kinds of joint model, the training algorithm and the problems through the processing will be introduced in detail. 　　Key words：Chinese Word Segmentation; Chinese Part-of-speech Tagging; Joint Model