基于dbscan优化算法的web文本聚类研究-计算机应用技术专业论文.docxVIP

摘 摘 要 随着Internet和Web技术的飞速发展，Web已经成为一个海量动态的信息资源 库：而文本是信息最重要的载体，研究表明信息有80％包含在文本中。因此人们迫 切需要能够从Web文本数据中快速有效地发现资源和知识的工具，而Web文本聚 类是一种有效的文本挖掘方法，也是Web数据挖掘中的核心技术之一。本文针对 Web文本聚类展开研究，主要工作如下： (1)对Web文本聚类关键技术进行概述，主要分析了网页采集、文本去噪、分 词、文本表示和特征降维等文本预处理的相关技术，其中针对TF．IDF方法的不足， 提出一种改进的特征项权值计算方法SDI．TF．IDF； (2)在基于向量空间模型中使用的文本相似性度量方法基础上，分析了Web文 本聚类算法，对几种典型的聚类方法从多个方面作以比较，并介绍Web文本聚类评 价标准； (3)分析了传统的DBSCAN算法思想的局限性，针对建立R车一树复杂耗时及 DBSCAN算法对非均匀数据集聚类效果差等缺点，在目前现有的IF．DBSCAN算法 的基础上，结合哈希表邻域查询法和核函数聚类，提出了对DBSCAN算法的改进 优化方法； (4)最后对采集到的Web文本进行聚类，实验验证了优化算法具有良好的聚类 结果。 关键字：Web文本聚类、向量空间模型、DBSCAN、哈希表法、核函数聚类 AbstractAs Abstract As the rapid development of Internet and Web technology,Web has already become a massive dynamic information resource database．Text is the most important carrier of information，and the research shows that 80％of the information has been contained in the text．So people urgently need to discover tools of resources and knowledge from the Web text data rapidly and efficiently．While Web text clustering is not only an effective method of text mining，but also one of the core technologies in Web data mining．In this thesis，the research on Web text clustering is conducted as follows： (1)Firstly,this thesis paper has made deep research on the key technology of Web text clustering and discussed the related technologies of text pretreatrnent，such as homepage gathered，text denoised and segemented，text expressed，feather dimension reduction and SO on．For the disadvantages of TFIDF,the SDI—TFIDF is presented to computer the weight of the features． (2)Secondly,it has also introduced the method of text similarity measure in the vector space model，analyzed the Web text clustering algorithms，compared several typical cluster methods from many aspects，and introduced the evaluation standards of text clustering； (3)Thirdly,I introduce the thought of traditional DBSCAN algorithm and analyze its limitations．Based on the IF-DBSCAN algorithm，aiming at some defects such as establishing complicated R*-tree time—consuming