Google云计算解决方案6_分布式文件系统.ppt

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * ? Spinnaker Labs, Inc. Local Caching - Tradeoffs Shared database files do not work well on this system Does not support write-through to shared medium ? Spinnaker Labs, Inc. Replication AFS allows read-only copies of filesystem volumes Copies are guaranteed to be atomic checkpoints of entire FS at time of read-only copy generation Modifying data requires access to the sole r/w volume Changes do not propagate to read-only copies ? Spinnaker Labs, Inc. AFS Conclusions Not quite POSIX Stronger security/permissions No file write-through High availability through replicas, local caching Not appropriate for all file types ? Spinnaker Labs, Inc. The Google File System ? Spinnaker Labs, Inc. Motivation Google needed a good distributed file system Redundant storage of massive amounts of data oncheap and unreliable computers Why not use an existing file system? Google’s problems are different from anyone else’s Different workload and design priorities GFS is designed for Google apps and workloads Google apps are designed for GFS ? Spinnaker Labs, Inc. Assumptions High component failure rates Inexpensive commodity components fail often “Modest” number of HUGE files Just a few million Each is 100MB or larger; multi-GB files typical Files are write-once, mostly appended to Perhaps concurrently Large streaming reads High sustained throughput favored over low latency ? Spinnaker Labs, Inc. GFS Design Decisions Files stored as chunks Fixed size (64MB) Reliability through replication Each chunk replicated across 3+ chunkservers Single master to coordinate access, keep metadata Simple centralized management No data caching Little benefit due to large data sets, streaming reads Familiar interface, but customize the API Simplify the problem; focus on Google apps Add snapshot and record append operations ? Spinnaker Labs, Inc. GFS Client Block Diagram GFS Architecture Single master Mutiple chunkservers …Can anyone see a po