基于dht的分布式价格搜索引擎研究-计算机技术专业论文.docxVIP

基于 基于 DHT 的分布式价格搜索引擎研究 西华大学硕士学位论文 西华大学硕士学位论文 PAGE PAGE IV PAGE PAGE III Abstract In recent years, with the diversification of network resources and peoples demand for information on the exclusive domain, the study on vertical search engine becomes more and more popular. The search oriented price is one of them. However, the existing price search engines are almost always based on centralized, when a large number of users request at the same time, central server will become a bottleneck and prone to become single points of failure. With the continuous expansion of network scale, research on distributed vertical search becomes more and more important. In this paper, we combined P2P technology and the vertical search engine, and designed a DHT-based distributed price search engine, discussed the focused crawlers crawling strategy, used URL rules to judgment the relevance of webpage theme, and used XPath technology to extract web information. Then we mainly discussed how to use the DHT ideology to achieve the index’s build and distributed storage, avoiding problems that the centralized index may occur effectively . Finally, aiming at the problem of unclear and confused that presented by the existing price search engines’ search results, this paper presents an idea of clustering the search results. Through the research and analysis of the existing clustering algorithms, in this paper we improved the k-means algorithm, and clustered the search results with the improved algorithm, making the similarity of the documents in the same cluster is high, and the similarity of the documents between clusters is low. Then each cluster are described by class label, users only need to browse the class label of interest to him without the need to browse all of the results returned, which saved the browsing time and lookup time greatly. Key Words: vertical search engine; P2P; focused crawler; DHT; distributed index; clustering algorithm 目 录 摘 要I Abstract II 目 录III 1 绪论1 1.1 课题研