A Hybrid MPI Design using SCTP and iWARP.ppt

A Hybrid MPI Design using SCTP and iWARP Distributed Systems Group Mike Tsai, Brad Penoff, and Alan Wagner Department of Computer Science University of British Columbia Vancouver, Canada April 14, 2008 A Hybrid Message Passing Interface Design using theStream Control Transmission Protocol and the Internet Wide Area Remote Direct Memory Access Protocol Distributed Systems Group Mike Tsai, Brad Penoff, and Alan Wagner Department of Computer Science University of British Columbia Vancouver, Canada April 14, 2008 Research Background SCTP – Stream Control Transmission Protocol IETF standardized transport protocol for IP Can be used anywhere TCP or UDP are used Additional features SCTP and MPI middleware LAM (unreleased)? MPICH2 (1.0.5 and on) ch3:sctp Open MPI SCTP BTL (in v1.3 trunk)? Hardware acceleration techniques for IP Protocol offload OS bypass Zero copy RDMA 10 GigE How would these look for SCTP? Are there benefits here for using SCTP? State-of-the-Art Networking iWARP - Internet Wide Area RDMA protocol IETF standard for RDMA over IP Use RDMA, point-to-point, or a mix? “Why Compromise?” (G. Shainer @ HPCW)? Depending on the application, use whichever is best. For MPI middleware, who decides what’s best? Story/motivation The programmer! Contribution Hybrid MPI with functional decomposition lets the programmer decide: Let RMA use RDMA Let other communications use point-to-point Explore SCTP’s use within iWARP Extended OSC userspace software iWARP, making many internal OSC changes iWARP : DDP LLP Direct Data Placement Fragments messages Reassembles segments Segments self-contained Data delivery and placement separation Out-of-order delivery Requires LLP to: Keep segment boundaries Be reliable Take a strong checksum iWARP : LLP = MPA over TCP Message PDU Aligned Message framing DDP segment vs. TCP stream Markers for out-of-order For middlebox fragmentation Stronger checksum … is a complex layer (majority of OSC code)! … can lead to non-compliant T