后缀树和其在中文文本聚类中的应用探索.pdfVIP

后缀树及其在中文文本聚类中的应用探索 摘 要 本文首先对研究后缀树聚类算法的背景 目标和意义进行了论述 并与传 统的聚类算法进行了对比 列举了一些传统聚类算法的失败案例 然后结合自 行开发的后缀树生成过程演示程序 对后缀树聚类算法的原理 流程 特点以 及优势进行了深入的剖析 并结合中文的特点 设计出了应用于中文的 STCC suffix tree clustering for Chinese 算法的流程 为了对流程的合理性进行测试 又自行开发了针对单篇文档的高频短语发现器和针对多篇文档的公共短语发现 器 并以主题相似的文档集 不同主题的文档集和主题一致的网页文档集作为 实验语料 根据STCC 聚类规则 并借助这两个工具进行了大量的实验和分析 最后得出后缀树聚类算法对中文文档的聚类是有效的 为今后该领域的研究提 供了一个新的思路 关键词 后缀树 短语 聚类 短语簇 公共短语 ii Suffix Tree and Its Application to Chinese Text Clustering Abstract Firstly, this paper discusses the background, purpose and significance of the study of suffix tree clustering algorithm, and then contrasts suffix tree clustering algorithm with traditional methods, enumerates some failure case of traditional ones. Secondly, with the demo program of suffix tree growing, this paper analyzes the principle, flow, characteristic and merit of suffix tree algorithm, and then according to the characteristic of Chinese, the paper designs the flow of STCC suffix tree clustering for Chinese . Thirdly, in order to prove the rationality of the flow of STCC, the paper designs the frequent phrase finder for single document and the common phrase finder for multiple documents, and then uses the document set of similar topic, document set of various topics and web document set of same topic as the materials for the experiment. Fourthly, according to the rule of STCC, it does some experiments and works out plenty of analyses with these two tools. At last, it draws the conclusion that STCC is efficient for Chinese, so this paper provides a new path for the study of this field. Key words: suffix tree, phrase, clustering, cluster of phrase, common