ROAR Increasing the Flexibility and Performance of….pdfVIP

ROAR: Increasing the Flexibility and Performance of Distributed Search Costin Raiciu Felipe Huici Mark Handley University College London NEC Europe, Heidelberg University College London c.raiciu@cs.ucl.ac.uk Felipe.Huici@nw.neclab.eu m.handley@cs.ucl.ac.uk David S. Rosenblum University College London d.rosenblum@cs.ucl.ac.uk ABSTRACT To search the web quickly, search engines partition the web index over many machines, and consult every partition when answering a query. To increase throughput, replicas are added for each of these machines. The key parameter of these algorithms is the trade-off between replication and partitioning: increasing the partitioning level improves query completion time since more servers handle the query, but may incur non-negligible startup costs for each sub- query. Finding the right operating point and adapting to it can sig- nificantly improve performance and reduce costs. We introduce Rendezvous On a Ring (ROAR), a novel distributed algorithm that enables on-the-fly re-configuration of the partition- ing level. ROAR can add and remove servers without stopping the Figure 1: Basic Distributed Rendezvous system, cope with server failures, and provide good load-balancing even with a heterogeneous server pool. We demonstrate these claims each query reaches machines that between them hold all the data. using a privacy-preserving search application built upon ROAR. We call this class of algorithms distr