并行程序设计10并行算法及性能分析.pptVIP

Matrix-Matrix Multiplication Consider the problem of multiplying two n x n dense, square matrices A and B to yield the product matrix C =A x B. The serial complexity is O(n3). A useful concept in this case is called block operations. In this view, an n x n matrix A can be regarded as a q x q array of blocks Ai,j (0 ≤ i, j q) such that each block is an (n/q) x (n/q) submatrix. In this view, we perform q3 matrix multiplications, each involving (n/q) x (n/q) matrices. Matrix-Matrix Multiplication Consider two n x n matrices A and B partitioned into p blocks Ai,j and Bi,j (0 ≤ i, j ) of size each. Process Pi,j initially stores Ai,j and Bi,j and computes block Ci,j of the result matrix. Computing submatrix Ci,j requires all submatrices Ai,k and Bk,j for 0 ≤ k . All-to-all broadcast blocks of A along rows and B along columns. Perform local submatrix multiplication. Matrix-Matrix Multiplication The two broadcasts take time The computation requires multiplications of sized submatrices. The parallel run time is approximately Major drawback of the algorithm is that it is not memory optimal. Matrix-Matrix Multiplication: Cannons Algorithm In this algorithm, we schedule the computations of the processes of the ith row such that, at any given time, each process is using a different block Ai,k. These blocks can be systematically rotated among the processes after every submatrix multiplication so that every process gets a fresh Ai,k after each rotation. Matrix-Matrix Multiplication: Cannons Algorithm Matrix-Matrix Multiplication: Cannons Algorithm Perform local block multiplication. Each block of A moves one step left and each block of B moves one step up (again with wraparound). Perform next block multiplication, add to partial result, repeat until all blocks have been multiplied. Matrix-Matrix Multiplication: Cannons Algorithm In the alignment step, since the maximum di