Hybrid programming MPI+OpenMP（混合编程MPI + OpenMP）.pdfVIP

Hybrid programming MPI+OpenMP Gabriele Fatigati - g.fatigati@cineca.it Supercomputing Group data The hybrid model  Multi-node SMP (Symmetric Multiprocessor) connected by and interconnection network.  Each node is mapped (at least) one process MPI and OpenMP threads more. MPI vs. OpenMP  Pure MPI Pro: Pure OpenMP Pro: High scalability Easy to deploy (often) High portability Low latency No false sharing Implicit communications Scalability out-of-node Coarse and fine granularity  Pure MPI Con: Dynamic Load balancing Hard to develop and debug. Pure OpenMP Con: Explicit communications Only on shared memory machines Coarse granularity Intranode scalability Hard to ensure load balancing Possible long waits for unlocking data Undefined thread ordering 3 Why hybrid?  MPI+OpenMP hybrid paradigm is the trend for clusters with SMP architecture. Elegant in concept: use OpenMP within the node and MPI between nodes, in order to have a good use of shared resources.  Avoid additional communication within the MPI node.  OpenMP introduces fine-granularity.  The two-level parallelism introduces other problems  Some problems can be reduced by lowering MPI procs number  If the problem is suitable, the hybrid approach can have better performance than pure MPI or OpenMP codes. 4 Optimizing the