Combining MPI and OpenMP参考.ppt

Parallel Programmingin C with MPI and OpenMP Michael J. Quinn Chapter 18 Combining MPI and OpenMP Outline Advantages of using both MPI and OpenMP Case Study: Conjugate gradient method Case Study: Jacobi method C+MPI vs. C+MPI+OpenMP Why C + MPI + OpenMPCan Execute Faster Lower communication overhead More portions of program may be practical to parallelize May allow more overlap of communications with computations Case Study: Conjugate Gradient Conjugate gradient method solves Ax = b In our program we assume A is dense Methodology Start with MPI program Profile functions to determine where most execution time spent Tackle most time-intensive function first Result of Profiling MPI Program Code for matrix_vector_product Adding OpenMP directives Want to minimize fork/join overhead by making parallel the outermost possible loop Outer loop may be executed in parallel if each thread has a private copy of tmp and j #pragma omp parallel for private(j,tmp) for (i=0; iBLOCK_SIZE(id,p,n); i++) { User Control of Threads Want to give user opportunity to specify number of active threads per process Add a call to omp_set_num_threads to function main Argument comes from command line omp_set_num_threads (atoi(argv[3])); What Happened? We transformed a C+MPI program to a C+MPI+OpenMP program by adding only two lines to our program! Benchmarking Target system: a commodity cluster with four dual-processor nodes C+MPI program executes on 1, 2, ..., 8 CPUs On 1, 2, 3, 4 CPUs, each process on different node, maximizing memory bandwidth per CPU C+MPI+OpenMP program executes on 1, 2, 3, 4 processes Each process has two threads C+MPI+OpenMP program executes on 2, 4, 6, 8 threads Results of Benchmarking Analysis of Results C+MPI+OpenMP program slower on 2, 4 CPUs because C+MPI+OpenMP threads are sharing memory bandwidth, while C+MPI processes are not C+MPI+OpenMP programs faster on 6, 8 CPUs because they have lower communication cost Case Study: Jacobi Method Begin with C+MPI program