HDFS架构设计指南.pdfVIP

HDFSArchitectureGuide

byDhrubaBorthakur

Tableofcontents

1Introduction3

2AssumptionsandGoals3

2.1HardwareFailure3

2.2StreamingDataAccess3

2.3LargeDataSets3

2.4SimpleCoherencyModel3

2.5“MovingComputationisCheaperthanMovingData”4

2.6PortabilityAcrossHeterogeneousHardwareandSoftwaretforms4

3NameNodeandDataNodes4

4TheFileSystemNamespace5

5DataReplication6

5.1Replicacement:TheFirstBabySteps7

5.2ReplicaSelection7

5.3Safemode8

6ThePersistenceofFileSystemMetadata8

7TheCommunicationProtocols9

8Robustness9

8.1DataDiskFailure,HeartbeatsandRe-Replication9

8.2ClusterRebalancing9

8.3DataIntegrity9

8.4MetadataDiskFailure10

8.5快照.10

9数据组织.10

9.1数据块.10

9.2阶段处理.11

9.3管道化.11

10可性.11

10.1文件系统外壳（FSShell）.12

10.2DFSAdmin.12

10.3浏览器界面.12

11空间回收.12

11.1文件删除和恢复.12

11.2减少副本因子.13

12参考文献.13

8.5Snapshots10

9DataOrganization10

9.1DataBlocks10

9.2Staging11

9.3ReplicationPipelining11

10Accessibility11

10.1FSShell12

10.2DFSAdmin12

10.3BrowserInterface12

11SpaceRection12

11.1FileDeletesandUndeletes12

11.2DecreaseReplicationFactor13

12References13

1引言

H