Floyd’s Algorithm.ppt

Parallel Programmingwith MPI and OpenMP Michael J. Quinn Chapter 6 Floyd’s Algorithm Chapter Objectives Creating 2-D arrays Thinking about “grain size” Introducing point-to-point communications Reading and printing 2-D matrices Analyzing performance when computations and communications overlap Outline All-pairs shortest path problem Dynamic 2-D arrays Parallel algorithm design Point-to-point communication Block row matrix I/O Analysis and benchmarking All-pairs Shortest Path Problem Floyd’s Algorithm Why It Works Dynamic 1-D Array Creation Dynamic 2-D Array Creation Designing Parallel Algorithm Partitioning Communication Agglomeration and Mapping Partitioning Domain or functional decomposition? Look at pseudocode Same assignment statement executed n3 times No functional parallelism Domain decomposition: divide matrix A into its n2 elements Communication Agglomeration and Mapping Number of tasks: static Communication among tasks: structured Computation time per task: constant Strategy: Agglomerate tasks to minimize communication Create one task per MPI process Two Data Decompositions Comparing Decompositions Columnwise block striped Broadcast within columns eliminated Rowwise block striped Broadcast within rows eliminated Reading matrix from file simpler Choose rowwise block striped decomposition File Input Pop Quiz Point-to-point Communication Involves a pair of processes One process sends a message Other process receives the message Send/Receive Not Collective Function MPI_Send Function MPI_Recv Coding Send/Receive Inside MPI_Send and MPI_Recv Return from MPI_Send Function blocks until message buffer free Message buffer is free when Message copied to system buffer, or Message transmitted Typical scenario Message copied to system buffer Transmission overlaps computation Return from MPI_Recv Function blocks until message in buffer If message never arrives, function never returns Deadlock Deadlock: process waiting for a condition that will never become true Easy to