基于层次的聚类.ppt

邀请话题 特征之间的相关性问题（SVD） 样本之间的推荐关系的利用。（Hits 算法） * /1354/548130658_b776b25bf6.jpg?v=0 * Web是一个有向图 * HTML 4.01 Specification: /TR/1999/REC-html401html40.txt * Given that the bandwidth for conducting crawls is neither infinite nor free, it is becoming essential to crawl the Web in not only a scalable, but efficient way, if some reasonable measure of quality or freshness is to be maintained. [1] ^ a b Edwards, J., McCurley, K. S., and Tomlin, J. A. (2001). An adaptive model for optimizing performance of an incremental web crawler. In Proceedings of the Tenth Conference on World Wide Web: 106–113. Hong Kong: Elsevier Science. doi:10.1145/371920.371960. . * Some pages here borrowed from /~zives/cis555/ It’s a nice course! * site:() baidu report 90,300 pages Google report 43,000 pages * 中国 日本 文化 -留学生 site:/ Baidu report 38 results Google report 908 results * incidence matrix 关联矩阵 * Linked lists generally preferred to arrays Dynamic space allocation Insertion of terms into documents easy Space overhead of pointers * How to build inverted index, 1M*1K terms document collection, There are 1G postings, can’t do it in memory. 讨论? * Queries using AND, OR and NOT together with query terms Views each document as a set of words Is precise: document matches condition or not. Primary commercial retrieval tool for 3 decades. Professional searchers (e.g., Lawyers) still like Boolean queries: You know exactly what you’re getting. 00/uhtbin/cgisirsi/nLEYr4CC23/%E5%8C%97%E5%A4%A7%E4%B8%AD%E5%BF%83%E9%A6%86/201190538/2/1000 * Thus far, our queries have all been Boolean Docs either match or not OK for expert users with precise understanding of their needs and the corpus Not good for (the majority of) users with poor Boolean formulation of their needs Most users don’t want to wade through 1000’s of results – cf. use of web search engines * Document is a vector: a column below Bag of words model WARNING: In a lot of IR literature, “frequency” is used to mean “count” Th