毕业设计（论文）-基于规则的分词算法研究与设计.doc

基于规则的分词算法研究与设计 摘要：中文作为人类沟通交流、传递信息的主要语言工具之一，那么中文信息的处理在信息领域就显得非常重要。中文分词是中文信息处理的基本也是最重要的组成部分之一，它的成功与否直接关系到中文信息处理领域能否取得重大突破。 中文分词是将中文的一段话或者一句话切分成一个一个单独的词，由于中文的词典没有明确规定词的定义，不像英文的单词之间有明确的分隔符，因此要实现中文的准确分词要比英文分词复杂得多也困难得多。目前，主要的分词算法有这么一些：正向最大匹配算法（MM）、逆向最大匹配算法（RMM）、最小切分算法、最佳匹配算法等。 本文先对一些常用的经典算法进行了分析与研究，然后根据任务要求，设计了一个简单的分词系统。系统主要分为三个部分：词典构造与载入、匹配词语、输出结果。本系统做出的改进有：减少了中文标点符号所引起的分词歧义；词典存储采用容器方式调入内存，提高分词效率。 关键字：中文分词；词典；最大匹配  Researching and Designing of Words Segmentation Algorithm based on Rules Abstract:Chinese as one of the main language tools for human communication and communicate information, so Chinese information processing in the information field is very important. The Chinese word segmentation is the basic and one of the most important part of Chinese information processing, the success directly related to Chinese information processing field can get significant breakthrough. The Chinese word segmentation is a passage into Chinese or words cut into a single word, since Chinese dictionary definition of the word did not make clear a regulation, dont like English words have clear between separator, so to achieve Chinese word segmentation of accurate than English word segmentation is much more complex and much more difficult. At present, the main parting-words arithmetic has few:maximal matching algorithm, reverse maximal matching algorithm, minimum segmentation algorithm, optimal matching algorithm and so on. This paper first to some common classical algorithms are analyzed and studied, according to the mission requirements, design a simple word segmentation system. System can be divided into three parts: dictionary structure and load, matching words, output the results. This system has made improvements: reduce caused Chinese punctuation word segmentation ambiguity; dictionary by a gelatinous memory storage containers way, improve efficiency . Keyword: Chinese word segmentation, dictionary, maximal matching algorithm  目 录 第1章 绪论 1 1.1 课题背景 1 1.2 研究目的与意义 2