理论科研成果垂直搜索引擎的研究与实现-计算机技术专业毕业论文.docxVIP

理论科研成果垂直搜索引擎的研究与实现Research 理论科研成果垂直搜索引擎的研究与实现 Research and Implementation of Theoretical Research Search Engine Abstra ct The progress of information technology has caused the massive growth of data．The huge amount of information data is distributed on every server in the world， and the information data is rapidly growing and updated．Vast amounts of information and data for Intemet users to provide rich information at the same time，also makes it difficult to extract the information they need，to obtain accurate information is like looking for a needle in a haystack．Although there are Bing，Google，Baidu and other outstanding search engines，but due to the huge amount of data on the Intemet，search engines do not collect all the information pages，not timely maintenance and update not effectively understand the user’S query needs，the accuracy of the query retums the results is not high，timeliness is not strong，high repetition rate．Therefore，the to search engine technology proposed new demand， search engine system must be able to provide users with more timely， more detailed， more accurate and deeper information， SO the vertical search engine technology came into being． This paper studies and implements a vertical search engine， which is based on the research results of specific basic theory．The basic principle of the research，the basic principle of the vertical search engine，the core technology and the process of web crawling， design and implementation of the information collection module， crawler crawling module， information indexing module， query results retrieval module， interaction module．In which the web page analysis module to achieve the function of the URL elimination， compared with two kinds ofweb page denoising scheme，the merit based on the function， and completed the web page tO re work．At last，the existing problems and the future dire：ction of the research are discussed． Key Words：Theoretical Research Results；Vertical Search Engine；Web Crawler；Lucene； Web Pag