王连平Pregel讲解.ppt

王 连 平 14210-1 2014201673 Example: SSSP – Parallel BFS in Pregel 0 8 5 11 7 10 5 2 3 2 1 9 7 4 6 9 14 13 15 Example: SSSP – Parallel BFS in Pregel 0 8 5 9 7 10 5 2 3 2 1 9 7 4 6 Example: SSSP – Parallel BFS in Pregel 0 8 5 9 7 10 5 2 3 2 1 9 7 4 6 13 Example: SSSP – Parallel BFS in Pregel 0 8 5 9 7 10 5 2 3 2 1 9 7 4 6 Differences from MapReduce Graph algorithms can be written as a series of chained MapReduce invocation Pregel Keeps vertices edges on the machine that performs computation Uses network transfers only for messages MapReduce Passes the entire state of the graph from one stage to the next Needs to coordinate the steps of a chained MapReduce Outline Introduction Computation Model Writing a Pregel Program System Implementation Experiments Conclusion Future Work C++ API Writing a Pregel program Subclassing the predefined Vertex class Override this! in msgs out msg Example: Vertex Class for SSSP Outline Introduction Computation Model Writing a Pregel Program System Implementation Experiments Conclusion Future Work System Architecture Pregel system also uses the master/worker model Master Maintains worker Recovers faults of workers Provides Web-UI monitoring tool of job progress Worker Processes its task Communicates with the other workers Persistent data is stored as files on a distributed storage system (such as GFS or BigTable) Temporary data is stored on local disk Execution of a Pregel Program Many copies of the program begin executing on a cluster of machines The master assigns a partition of the input to each worker Each worker loads the vertices and marks them as active The master instructs each worker to perform a superstep Each worker loops through its active vertices computes for each vertex Messages are sent asynchronously, but are delivered before the end of the superstep This step is repeated as long