并行计算介绍.pptVIP

Introduction to Parallel Programming MapReduce Serial vs. Parallel Programming In the early days of computing, programs were serial, that is, a program consisted of a sequence of instructions, where each instruction executed one after the other. It ran from start to finish on a single processor. Serial vs. Parallel Programming Parallel programming developed as a means of improving performance and efficiency. In a parallel program, the processing is broken up into parts, each of which can be executed concurrently. The instructions from each part run simultaneously on different CPUs. These CPUs can exist on a single machine, or they can be CPUs in a set of computers connected via a network. Why do it? Not only are parallel programs faster, they can also be used to solve problems on large datasets using non-local resources. When you have a set of computers connected on a network, you have a vast pool of CPUs, and you often have the ability to read and write very large files (assuming a distributed file system is also in place) How to do it? The first step in building a parallel program is identifying sets of tasks that can run concurrently and/or partitions of data that can be processed concurrently. How to do it? Sometimes its just not possible. Consider a Fibonacci function: Fk+2 = Fk + Fk+1 A function to compute this based on the form above, cannot be parallelized because each computed value is dependent on previously computed values. How to do it? A common situation is having a large amount of consistent data which must be processed. If the data can be decomposed into equal-size partitions, we can devise a parallel solution. Consider a huge array which can be broken up into sub-arrays…. How to do it? Implementation Strategy:Master/Worker The MASTER: initializes the array and splits it up according to the number of available WORKERS sends each WORKER its subarray receives the results from each WORKER The WORKER: receives the subarray from the