On-deand Grid Storage Using Scavenging.ppt

Outline Grid Storage Fabric Background The Evolving Computing Landscape—An Analogy Storage Scavenging of User Desktop Workstations Use Cases Related Work and Design Choices Architecture Storage Layer Management Layer Current Status Grid Storage Fabric Background Scientific discoveries driven by analyses of massively distributed, bulk data. Proliferation of high-end mass storage systems, SANs and datacenters Providers such as IBM, HP, Panasas, etc. Merits: Excellent price/performance ratio Good storage speeds and access control Support intelligent parallel file systems Optimized for wide-area, bulk transfers Reliability! Successful demonstration in production Grids: DOE Science Grid, Earth System Grid, TeraGrid, etc. Drawbacks: Increasing deployment/maintenance/administrative costs Specialized software and central points of failure Costs and specialized features prohibit wider acceptability and limits to select few research labs organizations Aforementioned production Grids are hardly half-a-dozen sites…!! The Evolving HPC Landscape Computing fabric for the Grid: Storage Fabric…?? Storage Scavenging of User Desktop Workstations Harnessing collective storage potential of individual workstations ~ Harnessing idle CPU cycles Why Storage Scavenging can be viable? Economics of buying gigabytes of storage is increasingly affordable Space usage to Available storage ratio is significantly low Increasing numbers of workstations are online most of the time Even a modest contribution (Contribution Available) can amass collective, staggering aggregate storage! Concerns: Vagaries of volatility… Question of Trust: datasets on arbitrary user workstations Performance of such aggregate storage Use Cases Related Work and Design Choices Related Work: Network/Distributed File Systems (NFS, LOCUS) Parallel File Systems (PVFS, XFS) Serverless File Systems (FARSITE, xFS, GFS) Peer-to-Peer Storage (OceanStore, PAST, CFS) Grid Storage Services (LegionFS, SRB, IBP, SRM, GASS) Arch