基于潜在语义分析的BBS文档Bayes鉴别器-中国科技论文在线.PDFVIP

__________________________________________________________________________ 基于潜在语义分析的BBS 文档Bayes 鉴别器1 1) 1) 2) 1) 刘昌钰 唐常杰 杜永萍 郭颖 1)(四川大学计算机系，成都，610064) 2)(山西大学计算机系，太原，030006) 摘 要：对电子公告栏（BBS）文档进行鉴别已成为信息安全技术的重要内容之一。本文融合了数据挖 掘技术、数理统计技术和自然语言理解技术，提出了基于潜在语义分析与Bayes 分类的BBS 文档鉴别方 法 ( Bayes Discriminator based on Latent Semantic Analysis, BDLSA).利用自然语言处理技术从训练文档中 抽取典型短语集；通过潜在语义分析进行典型短语同义归约，应用关联规则采掘技术提高典型短语间的 独立性；用Bayes 分类器对BBS 文档进行鉴别。本文还对影响系统的关键参数进行了大量的讨论和测试， 实验表明新提出的方法对于BBS 文档的鉴别是可行而有效的。 关键词：数据挖掘，关联规则，贝叶斯分类，潜在语义分析，BBS。 中图法分类号：TP?? Bayes Discriminator for BBS Documents based on Latent SemanticAnalysis LIU Changyu1) TANG Changjie 1) DU Yongping2) GUO Ying1) 1) (Department of Computer Science, Sichuan University, Chengdu，610064) 2) (Department of Computer Science, Shanxi University, Taiyuan，030006) Abstract: With the rapid development of Internet, the abuse and misuse of BBS become a social problem of information pollution and call on the demand to the discrimination techniques for BBS document. Borrowing the techniques from data mining, probability-statistics and Natural Language Understanding, this paper proposes a new discrimination method for BBS document, called Bayes Discrimination based on Latent Semantic Analysis (BDLSA). The main steps of new method includes (1) Make typical phrase set by extracting the typical sentences from training documents in preprocessing stage with natural language understanding techniques. (2). Apply synonymy reduction on typical phrases by Latent Semantic Analysis (3) Discover the association rules between typical phrases to increase the independency of phrases so that the traditional Bayes discriminator works efficiently. (4) Discriminate BBS Document by Bayes cla