一种基于邻居规则分类算法的聚焦爬虫.pdfVIP

ISSN 1009-3044 E-maü: eduf@d nz$. ComputerKnowledge and TI邵阳ology 电脑知识与技术 http://www.命lZ$. Vol 3, No 4, May 20 7 Tel:+86-551 65690964 -种基于邻居规则分类算法的聚焦爬虫 敬炬，鲁红英 (成都理工大学信息科学与技术学院， 四川 成都6 ∞59) 捕要:判定链接锚文本与主题的相关皮、评估链接的优先级并过滤不相关的链接是实现聚焦爬行的关键。 通过判定链接 所在页而是否主题相关、是否是导航页面划分链接的类别，结合链接锚文本与主题的相似度，提出了一种基于邻居规则分 类算法评估链接优先级的聚焦爬虫。 该爬虫包括一个主题相关网页判别器、一个导航网页套J~J 器和一个基于邻居规则分 类算法的链接优先级评估器。 实验结采斗之明 ，基于邻居规则分类算法的聚焦爬虫比仅仅根据锚文本判定链接优先级的标 准聚焦爬虫具有更好的性能，因此更加适合用于信，忽检索。 关键词:信息检索;聚焦爬虫;邻居规则;分类算法;链接优先级 中图分类号: TP391 文献标i只码:A 文章编号:1009-3044(2017)14-0151-03 Focused Crawler ßased Oß CRN C lassi1icatioß Algorithm JING Xuan ,LU H ong Ying (College of lnfor11lation $cience Technology, C hengdu University of Technology, C hengdu 610059, C hina) Abstract: T he key of focused crawling is to assess similarity between anch or text and given topic, esti11late priority of links and 6 1 ter irrelevant links. A focllsed crawler based on C民N classi6ωtion algorith 11l is proposed in this work, w hich determin e category of links by deter11lining whether the page links located in relate given topic and whether the page is navigation web page. Priority of the links is estimated based their category and the similarity between anch or text and given topic. T he crawler consist of a rele vant page classi6er, a navigation page classi6er and a link priority evalllator based on C民N classi6cation algorithm. CI飞N(Classi6 cat:i on by R ule based Neighbors) classi6cation algorithm is introduced is this paper. Experiment reslll臼 shows that focllsed crawl er based on CRN classi6cation algorithm has better behavior than standard focllsed crawler only estimating link priority based on anchor text Key