On the strongFeasibilitystrong of Peer-to-Peer Web Indexing and Search.pdfVIP

On the Feasibility of Peer-to-Peer Web Indexing and Search Jinyang Li Boon Thau Loo Joseph M. Hellerstein M. Frans Kaashoek David Karger Robert Morris MIT Lab for Computer Science UC Berkeley jinyang@, boonloo, jmh @, kaashoek, karger, rtm@ Abstract search, including Gnutella [1] and KaZaA [4]. These systems use the simple and robust technique of flood- This paper discusses the feasibility of peer-to-peer ing queries over some or all peers. The estimated full-text keyword search of the Web. Two classes of number of documents in these systems is 500 mil- keyword search techniques are in use or have been lion [3]; documents are typically music files, and proposed: flooding of queries over an overlay net- searches examine only file meta-data such as title and work (as in Gnutella), and intersection of index lists artist. These systems have performance problems [6] stored in a distributed hash table. We present a sim- even with workloads much smaller than the Web. ple feasibility analysis based on the resource con- straints and search workload. Our study suggests Another class of P2P systems achieve scalabil- that the peer-to-peer network does not have enough ity by structuring the data so that it can be found capacity to make naive use of ei