CSE 160 ndash; Lecture 16.ppt

CSE 160 – Lecture 16 MPI Concepts, Topology and Synchronization So Far … We have concentrated on generic message passing algorithms We have used PVM to implement codes (simpler on workstations) MPI is a defacto standard for message passing and introduces some important concepts Message Addressing Identify an endpoint Use a tag to distinguish a particular message pvm_send(dest, tag) MPI_SEND(COMM, dest, tag, buf, len, type) Receiving recv(src, tag); recv(*,tag); recv (src, *); recv(*,*); What if you want to build a library that uses message passing? Is (src, tag) safe in all instances? Level O Issues Basic Pt-2-Pt Message Passing is straightforward, but how does one … Make it go fast Eliminate extra memory copies Take advantage of specialized hardware Move complex data structures (packing) Receive from one-of-many (wildcarding) Synchronize a group of tasks Recover from errors Start tasks Build safe libraries Monitor tasks … MPI-1 addresses many of the level 0 issues (but not all) A long history of research efforts in message passing P4 Chameleon Parmacs TCGMSG CHIMP NX (Intel i860, Paragon) PVM … And these begot MPI So What is MPI It is a standard message passing API Specifies many variants of send/recv 9 send interface calls Eg., synchronous send, asynchronous send, ready send, asynchronous ready send Plus other defined APIs Process topologies Group operations Derived Data types Profiling API (standard way to instrument MPI code) Implemented and optimized by machine vendors Should you use it? Absolutely! So What’s Missing in MPI-1? Process control How do you start 100 tasks? How do you kill/signal/monitor remote tasks I/O Addressed in MPI-2 Fault-tolerance One MPI process dies, the rest eventually hang Interoperability No standard set for running a single parallel job across architectures (eg. Cannot split computation between x86 Linux and Alpha) Communication Envelope (src, tag) is enough to distinguish messages however, there is a problem: How does one writ