云计算课件 Spark课件.pptVIP

Spark和MapReduce比较 iter. 1 iter. 2 . . . Input HDFSread HDFSwrite HDFSread HDFSwrite Input query 1 query 2 query 3 result 1 result 2 result 3 . . . HDFSread Spark MapR I/O and serialization can take 90% of the time Cache Cache Cache Transf T..2 Action RDD操作 – Resilient Distributed Dataset 所有的操作都是针对RDD，类似于MPPDB的技术实现：分布、并行、内存计算和压缩。优于MPP点在于毫秒级的调度，适用于复杂计算；逊于MPP点在于数据处理没有SQL方便和强大。 Spark 如何组成分布式网络 Your application SparkContext Cluster manager Worker Spark executor HDFS or other storage Worker Spark executor sc = new SparkContext f = sc.textFile(“…”)f.filter(…) .count()... Your prog