HDF5 Ttorial.ppt

HDF5 Tutorial SDSC Computing Institute July 26, 2005 Goals Introduce HDF5 Provide a basic knowledge of how data can be organized in HDF5 how it is used by applications To provide some examples of how to read and write HDF5 files in sequential and parallel modes Outline Sequential HDF5 (Elena Pourmal) Short Introduction “What is HDF5?” Overview of the HDF5 Data Model and I/O Library Introduction to HDF5 API Examples for file creation/write/read Parallel HDF5 (Albert Cheng) Overview of Parallel HDF5 design Setting up parallel environment Exampled for file creation/write/readWhat is HDF5? What is HDF5? File format for storing scientific data To store and organize all kinds of data To share data , to port files from one platform to another To overcome a limit on number and size of the objects in the file Software for accessing scientific data Flexible I/O library (parallel, remote, etc.) Efficient storage Available on almost all platforms C, F90, C++ , Java APIs Tools (HDFView, utilities) Example HDF5 file OverviewHDF5 Data Model  I/O Library HDF5 Data Model HDF5 file HDF5 file – container for storing scientific data Primary Objects Groups Datasets Additional means to organize data Attributes Sharable objects Storage and access properties HDF5 Dataset HDF5 dataset – data array and metadata Data array ordered collection of identically typed data items distinguished by their indices Metadata Dataspace – rank, dimensions, other spatial info about dataset Datatype Attribute list – user-defined metadata Special storage options – how array is organized and stored in the file Dataset ComponentsExample Matrix A(7,4,5) Dataspaces Dataspace – spatial info about a dataset Rank and dimensions Permanent part of dataset definition Subset of points, for partial I/O Needed only during I/O operations Apply to datasets in memory or in the file Sample Mappings between File Dataspaces and Memory Dataspaces Datatypes (array elements) Datatype – how to interpret a data eleme