parallel mxn communication using mpi-io.pptVIP

Parallel MxN  Communication Using MPI-I/O Felipe Bertrand, Yongquan Yuan, Kenneth Chiu, Randy Bramley Department Of Computer Science, Indiana University Supported by NSF Grants 0116050 and EIA-0202048 Department of Energys Office of Science SciDAC grants Motivation Interesting problems span multiple regimes of time, space, and physics. Multiphysics, multidiscipline: Climate models Combustion models Fusion simulation Protein simulation Community Climate System Model  CCSM CCSM models are parallel but the inter-model communication is serial Existing Approaches Refactor codes: integrate all components into a single, much larger program More efficient data sharing Large time investment in rewriting program Closer coordination between teams: more costly in development, testing and deployment Component models Simplify the overall application development. Complicate the efficient coordination and sharing of data between components MxN problem . Goals Scalably connect codes to create new multiphysics simulations Target existing and currently evolving codes Created by teams with disparate research interests Spanning multiple time scales, spatial domains, and disciplines Rapid prototyping/testing without extensive rewriting Use standard APIs and paradigms familiar to most application area scientists MPI I/O The MxN Problem Transfer data from a parallel program running on M processors to another running on N processors. M and N may differ May require complex all-to-all communications, data redistribution Solving the MxN Problem Existing solutions Use process 0 on all components Used by CCSM model Not scalable Read/Write through files Scalable if parallel I/O used Slow because involves hard drive read/write Our solution Use MPI I/O interface, create middleware to transfer data via network Treat application codes as software components Provide easy migration path for existing applications Solving the MxN Problem MPI-I/O defines an API for parallel I/O