互联网网页文本对象抽取实现技术本科生_毕业论文_毕业论文.doc

湖南大学毕业论文 第  PAGE 30 页 湖南大学毕业论文 第  PAGE 40 页 湖南大学软件学院 湖南大学软件学院 互联网网页文本对象抽取实现技术 摘 要 互联网中蕴含着大量的关于现实世界对象的结构化信息。为了能应对信息爆炸带来的严重挑战，抽取、集成网页上各式各样的文本对象信息，进行对象级别的搜索，迫切需要一些自动化的技术帮助人们在海量信息中迅速找到自己真正需要的信息。 网页文本对象抽取实现技术正是解决这个问题的一种方法。 本文以传统的信息抽取理论和方法为基础，针对目前热门的博客领域，提出了一种基于HTML特征和机器学习的博客正文抽取算法。在该算法中，研究了博客网页的特征，提出了一种基于HTML标签特征的网页分块算法，使用决策树算法对博客数据集进行统计训练，采用专门的统计工具WEKA对该算法???行了测试和评估，并总结出该算法的优点以及可以改进的地方。最后，展示了基于该博客正文抽取算法的博客搜索引擎Geeseek的系统结构和界面演示。该系统属于新型的垂直搜索引擎，能够对博客和博文进行快速有效的搜索。据了解，Geeseek也是目前国内高校中第一个博客搜索引擎。 关键词：互联网，信息爆炸，信息抽取，博客，HTML，机器学习，决策树，搜索引擎，Geeseek Implementation of text object extraction for Internet web pages Author: Zhang Hui Tutor: Lin Yaping Abstract Nowadays, there is a large number of semi-structural information which represents objects in the real world on the Internet. In order to deal with the severe challenge brought by information explosion, extract and integrate all kinds of text object information on web pages, and put up the object-level searching, it cries for the automated technologies to help people find the very information they really need among such a large number of information. The technology of text object extraction is just one of methods to solve this problem. Based on the traditional theory of Information Extraction and aiming at the blog domain, this paper puts forward an arithmetic implementing the extraction function for the text objects of blog articles with the HTML features and machine learning. In this arithmetic, it analyses the features of blog pages, introduces an arithmetic for web page partition basing on the HTML tag features, uses decision tree to do statistics and training on the blog data set, tests and evaluates this arithmetic using the expert statistical tool, WEKA, and summarizes the advantages as well as the points needing improving. Fina