MapReduce3.pptVIP

MapReduce:Simplified Data Processing on Large Clusters /~course/cs402/2009 Hongfei Yan School of EECS, Peking University 7/9/2009 What’s Mapreduce Parallel/Distributed Computing Programming Model Typical problem solved by MapReduce 读入数据: key/value 对的记录格式数据 Map: 从每个记录里extract something map (in_key, in_value) - list(out_key, intermediate_value) 处理input key/value pair 输出中间结果key/value pairs Shuffle: 混排交换数据 把相同key的中间结果汇集到相同节点上 Reduce: aggregate, summarize, filter, etc. reduce (out_key, list(intermediate_value)) - list(out_value) 归并某一个key的所有values，进行计算 输出合并的计算结果 (usually just one) 输出结果 Ma