high performance file io for the blue genestronglstrong supercomputer.pdfVIP

High Performance File I/O for The Blue Gene/L Supercomputer IBM T. J. Watson Research Ctr, Yorktown Hts, NY yuh,rsahoo,chowson,gheorghe,castanos,mgupta@ IBM System Tech. Group, Rochester, MN IBM Almaden Research Ctr, San Jose, CA jmor eira,jjparker@ engelspn@ MCS, Argonne Nat’l Lab., Argonne, IL rross,thakur,robl,gropp@ Abstract Parallel I/O plays a crucial role for most data-intensive applications running on massivelypar allel systems like Blue Gene/L that pr ovides the pr omise of delivering enormous computational capability. We designed and implemented a highly scalable par allel file I/O architecture for Blue Gene/L, which leverages the benefit of the hierarchical and functional partitioning design of the system software with separate computational and I/O cores. The architecture ex- ploits the scalability aspect of GPFS (General Parallel File System) at the backend, while using MPI I/O as an interface between the application I/O and thefile system. We demon- strate the impact of our high performance I/O solution for Blue Gene/L with a comprehensive evaluation that consists of a number of widely used par allel I/O benchmarks and I/O intensive applications. Our design and implementation is not only able to deliver at least one order of magnitude