(高级数据库)05-MapReduce.pptx

高级数据库及大规模存储技术;Agenda;1 Map Reduce编程模型;1.1 并行体系结构的分类;1.2 MapReduce;1.3 应对挑战;1.4 好主意;1.5 简单性;2.1 Map in Lisp (Scheme);2.2 基本假设;2.3 Distributed Grep;2.4 热身 (1/2);2.4 热身(2/2);2.5 MapReduce方案;2.5.1 MapReduce Dataflow;2.5.2 Example: Word Count;2.5.3 单词计数——map;2.5.4 单词计数——shuffle;2.5.5 单词计数——reduce;2.5.6 单词计数——Combiner(1/2);Combiner Function (2/2);2.5.7 Map+Reduce;2.6 函数式编程的根源;2.7 MapReduce;2.7.1 MapReduce图示;2.8 MapReduce “Runtime”;2.9 程序员的额外工作;更完整的MapReduce视图;2.10 Map/Reduce;3 Google MapReduce;3.1 GFS;3.1.1 GFS设计假设;3.1.2 体系结构;3.1.2 GFS体系结构;3.2 API;3.2.1 Files and Chunks;3.2.2 GFS Chunk 服务器;3.2.6 GFS 主节点;3.2.3 Create操作;Create;Create;Create;3.2.4 名字空间;3.2.5 Locking;3.2.6 Read操作;Read Operation;Read Operation;Read Operation;3.2.6 Write操作;Write;Write;Write;Write;Write;Write;Write under failure;Write under failure;Leases;Write with primary;Write with primary;Write with primary;Write with primary;Write with primary;Write with primary;Write with primary;3.3 MapRduce的执行过程;3.3.1 程序复制分发;3.3.2 主任务;3.3.3 Map阶段;3.3.4 缓存处理;3.3.5 数据排序;3.3.6 Reduce阶段;3.3.7 后处理;3.4 容错;3.4.1 主动式容错;3.4.2 输入错误的容错;3.4.3 worker失效的容错;3.4.4 Master失效的容错;3.5 状态监测;3.6 改进;Model is Widely Applicable;小结;How to use it