analysis- principles of search engine - crawling, links, rankings, etc.（分析-搜索引擎原理-爬行,链接,排名等。）.doc

Analysis: principles of search engine - crawling, links, rankings, etc. First on the map to simply look the search engine of the ‘three axes’: data collection -gt; pretreatment [Index] -gt; rankings. Data collection The data collection phase, the website from ho like desert world of the Internet to collect its own database for storage. One, grab maintenance strategy Face a lot of data to be processed, many issues need to be considered in advance, such as ‘instant capture’ data or ‘prior crawl’? When performing maintenance on the data is a ‘regular crawl’ (regularly crawl once great depth, replace the original data or ‘incremental crawl’ (in the original data as the foundation for inter-face? 2, Link Tracking We all know that spiders are crawling and crawling along the links page. How to quickly crawl to the relative importance of the user information, and to achieve broad coverage is undoubtedly a need to focus on search engines consider. First is the first one, how to crawl important information. Want to know this, home to understand is how people subjectively important to determine whether a page of (their first thought the next fact nothing less than the following situations: Pages of historical weight accumulation (domain names a long time, high quality, seniority, many people will refer to this page (external links pointing to a lot of people would refer to this page (reproduced or mirrored, this page is easy for users to quickly browse (level than shallow, there are often new content appears (updates, and more. In the link tracking stage, in fact, the information can be obtained only ‘This page is easy for users to quickly browse (level shallow’, the other information is not obtained. For information on the coverage, in fact, when the spiders follow links two strategies: the depth and breadth crawl crawl. Think about the hip also know that breadth crawl facilitates access to more information about the depth of crawling to