基于词典、规则的斯拉夫蒙古文词切分系统的研究.pdfVIP

基于词典、规则的斯拉夫蒙古文词切分系统的研究1 史建国 侯宏旭 飞龙 （内蒙古大学计算机学院，呼和浩特，010021 ） E-mail:tieshushjg@163.com 摘 要：斯拉夫蒙古文是蒙古国现行的文字，又称为西里尔蒙古文或新蒙古文。蒙古文词干和词缀包含着 大量信息，斯拉夫蒙古文词切分是斯拉夫蒙古文信息处理众多后续工作的基础。本文尝试了将词典和规则 结合的方法对斯拉夫蒙古文进行词切分。首先预处理部分蒙古文词，然后基于词典切分高频和部分不符合 规则的词。最后对剩余的词，用切分规则生成多个候选集词切分方案，然后在这些方案中选出最优方案。 通过两种方法的有机结合，发挥各自的优点，得到了性能较好的斯拉夫蒙古文词切分系统。 关键词：斯拉夫蒙古文；词切分；词典；规则 Researching of Slavic Mongolian Word Segmentation System Based on Dictionary and Rule SHI Jian-guo ,HOU Hong-xu, BAO Fei-long (College of Computer Science, Inner Mongolia University, Hohhot 010021,China) E-mail:tieshushjg@163.com Abstract: Slavic Mongolian is the current language in Mongolia,witch also known as Cyrillic Mongolian or new Mongolian.Mongolian stemming (root) and affixes contains a lot of information,Researching of our Slavic Mongolian word segmentation system is the basis of many follow-ups of the Slavic Mongolian information processing work. This article was a try in the Mongolian word segmentation combining dictionary based method with rule based method.We preprocess the part of the Slavic Mongolian words first,and then,do the Segmentation on high-frequency part and not consistent with rules part of Mongolian words with dictionary based method.Finally, process the remaining Mongolian words, first step is to generate multiple candidates using with the various rules of Mongolian,second step is to select the correct segmentation from the candidates.We combine the two different methods, play their respective advantages and get the Slavic Mongolian word segmentation system with excellent performance. Keywords: Slavic Mongolian; Word Segmentation; Dictionary ; Rule 1 引言 蒙古语是黏着性语言，黏着语语言是一种有时态变化的语言类型，通过在单词的词尾缀 接不同的词缀来