Web新闻热点发现系统的设计与实现-通信与信息系统专业论文.docxVIP

I I 摘要 过去几十年，互联网作为一种新兴事物，发展十分迅速，信息量也急剧增长。 随着门户网站、论坛、社区等平台日益成熟，网上短消息、即时通讯、博客、微博 等技术应用迅速普及，互联网逐渐成为网民自由发表言论、表达情绪的重要平台， 也更容易让网民关注的热点在短时间内发展成舆情。 这些引起网民广泛关注的热点在不法分子的推波助澜下很可能对社会造成重大 影响。因此，迫切需要在热点形成初期就进行适当的引导。本文设计并实现的 Web 新闻热点发现系统就是解决这个问题的基础。以 Web 新闻标题和正文分词结果为基 础，本文的亮点是对 Web 新闻特点进行了深入分析，针对重复转载新闻、热点演化 新闻和评论性新闻分别采取了不同处理思路，并选择了文档主题发现效果较好的基 于 SNN 密度文本聚类算法，以 Web 界面的形式呈现系统自动发现的热点。最后，以 样本新闻集人工分类的结果作为参照对系统热点发现的效果进行了准确率和召回率 的评估，准确率较高，召回率一般。 本文讨论了 Web 新闻热点发现系统的开发背景和整体需求，紧紧围绕 Web 新闻 的特点，提出了系统的功能层次结构，详细阐述了 Web 新闻热点发现流程每个环节 的具体设计思路和实现步骤，并对系统 Web 新闻热点发现的效果进行了测试和评估， 总体效果较好，也总结了一些经验。 关键词：热点发现, Web 新闻, 文本聚类, 向量空间模型, 奇异值分解 II II ABSTRACT During the past years, Web which is seen as a new media has developed rapidly and a large amount of information has also appeared. Many platforms are growing up day by day, such as portal sites, forums, communities. Many applications are spreading rapidly, such as short message, instant message, blog, micro-blogging. So the Web has become an important platform for all the cyber citizens to voice opinion and to show emotion freely. Besides the hot-topics cyber citizens concern can easily develop into public sentiment. These hot-topics are likely to make great influence to the social if they are used improperly by some bad persons. So it is urgent to take some guidance at the beginning of the shaping of the hot-topics. My work described in this thesis is just the foundation to achieve this goal. Based on the words split from the title and the content of the Webs news, the highlight of this thesis is a deep analysis of the feature of the Web news. I have adopted different way to deal with the corresponding type of Web news. Besides I have chosen a proper text clustering algorithm which is called shared nearest neighbor to find all the hot-topics from a certain collection of Web news using the form of Web page. Furthermore, compared with the result classified by the manual work, I have also taken a test which showed the high precision and the normal recall of