基于蚁群算法的主题爬虫技术分析与实现-analysis and implementation of topic crawler technology based on ant colony algorithm.docxVIP

analysis of the relevant theory, which focuses on analysis of ant colony algorithm and its implementation using Java language, followed by analysis of server logs as well as on the Web log mining.Focused crawler search strategies currently the main difficulties are:Subject to the Web crawler search space in the overall distribution of information resources is unknown, can not well predict the direction of crawling.The subject of reptiles at this stage thinking mostly through analysis of the link anchor text and theme relevant content strategy to guide the reptiles crawling, do not have the inspiring to guide policy.In order to give priority to high quality relevant web crawling, the researchers designed a number of heuristic strategies and related algorithms, although theres heuristic search strategy can make use of knowledge in certain areas of the distribution of information resources and make some kind of estimates of the extent by which to infer the approximate search direction, but this method of calculation capacity, and high computational complexity, is also not find it appropriate learning algorithm to guide the training process.This paper presents an ant colony algorithm based on the theme of crawler technology, mining from the Web log information in the users browsing path of a group, so that it can better guide the theme of reptiles. Finally, experimental realization of a theme reptiles, compared with themes based on ant colony algorithm and the traditional theme of reptiles difference between reptiles and concluded the subject based on ant colony algorithm technology to better guide the reptile theme of reptiles.Keywords: Vertical searchtopics crawlerPageRank algorithmant colony Heritrix独创性声明本人声明所呈交的学位论文是本人在导师指导下进行的研究工作及取得的 研究成果。据我所知，除了文中特别加以标注和致谢的地方外，论文中不包含其 他人已经发表或撰写过的研究成果，也不包含为获得成都理工大学或其他教 育机构的学位或证书而使用过的材料。与我一同工作的同志对本研究所做的任何 贡献均已在论文中作了明确的说明并表示谢意。学位论文作者签名：年月日学位论文版权使用授权书本学位论文作者完全了解成都理工大学 有关保留、使用学位论文的规定， 有权保留并向国家有关部门或机构送交论文的复印件和磁盘，允许论文被查阅和 借阅。本人授权