基于agent的专题搜索引擎爬虫的研究-计算机应用技术专业论文.docxVIP

江苏大学硕士学位论文ABSTRACT 江苏大学硕士学位论文 ABSTRACT With the extensive application ofⅥWnⅣtechnology,the traditional search engines ar e facing enormous challenges for owning lower in a recall rate，retrieval accuracy and not timely update always．They can not express good demand for users and show the users search results containing a substantial amount of information with unrelated to topic．At the same time，with an increasing number of different areas of customer, professional search engines able tO providing much more efficient retrieval are needed in tiers’specific industry． Taking advantage the unique method of specific site acquisition，topic-specific acquisition and digging web structure，Topic-Specific Search Engine can promote the efficiency of the entire recall rate，retrieval accuracy,and higher guarantee in its timeliness and professionalism，also provide better personalized services．In that way,it 啪be highly effective in identifying the specific areas of information and providing a unique retrieval services．So the network crawler design is the core of search engine．This paper is main about the crawler research of Agent-based Topic-Specific Search Engine and related key technologies，the major work of this paper are： 1．Based on the analysis of search engine technology,agent adaptive technology and machine learning,a Crawler Frame of Agent-based Topic-Specific Search is presented． 2．A Chinese Segmentation algorithm with cx／mbining word library and statistics is presented．Taking advantage of improved VSM(vector space model) and the use of improved Methods of web mining and content mining．a automatic text classification algorithm is designed based on support vector machine． 3．The search strategy is presented based on Q—learning algorithm，the algorithm is combining with web page evaluation technology and web link structure technology, with the help of adaptive agent，the algorithm can reduce the greed of the search to some extent and have a more effective manlier to choose web