2011-张旭东-bthesis-锚文本在搜索引擎中的存储及应用.docVIP

本科生毕业论文 题目：(中文) 锚文本在搜索引擎中的存储及应用 (英文) The Storage and Application of Anchor Text in Search Engine 姓 名： 张旭东二〇一一年五月二十七日 摘要 随着互联网的飞速发展，信息检索在实际生活中的作用越来越重要，在学术界也引起了研究学者的重视。互联网规模的不断扩大，使得作为网页之间的链接文本的锚文本的作用越来越大。针对锚文本在搜索引擎中的研究近十年才逐步开展起来的。锚文本在搜索引擎应用非常广泛，因此设计一种高效的锚文本的存储结构就显得非常重要。 本文在北京大学网络实验室天网组的智能中文搜索引擎平台PARADISE，对原有锚文本的存储结构进行了改进，设计了一种新的附加信息存储结构，这种结构额外存储了锚文本的附加信息，相比于以往的结构来讲又节省了存储空间。另外，本文使用锚文本新的存储结构在PARADISE中设计并实现了相应的打分模块，用于锚文本的检索。在TREC09 Web Track的Clueweb09B数据集上，同时使用PARADISE锚文本新旧存储结构和打分模块，进行了锚文本的构建索引和检索的对比实验，实验结果发现新的结构下锚文本的存储空间有了显著的减少，锚文本的检索效果相比于原先的检索方法也有了提高。 关键词：锚文本 PARADISE 存储 检索Abstract With the rapid development of Internet, information retrieval is playing a more and more important role in individual’s life and enterprise’s production, and has drawn attention in the academic circle. The ever-increasing scale of internet has revealed important effect of anchor text, which is text of hyperlinks between webpages. Research of anchor text in search engine has been just starting in recent decade. Since anchor text has a wide application in search engine domain, it’s essential to design an effective storage structure to store anchor text in search engine. This paper is based on the system of PARADISE(Platform for Applying, Researching And Developing Intelligent Search Engine), developed by Search Engine and Web Mining group in Network Laboratory, Peking University. We modified the original storage structure of anchor text in PARADISE and designed a new type of storage structure of extra information, which saves the storage space of anchor text compared to the old type, as well as store extra information of anchor text for retrieval. In addition, we designed the retrieval module of anchor text based on the new storage structure in PARADISE. Using the dataset of Clueweb09B in TREC09 Web Track, we conducted the experiment of both the storage and retrieval of anchor text. The experiment result showed that the storage space needed for new storage structure of an