SparkStreaming大规模准实时流式数据处理.ppt

Performance Can process 6 GB/sec (60M records/sec) of data on 100 nodes at sub-second latency Tested with 100 streams of data on 100 EC2 instances with 4 cores each * Comparison with Storm and S4 Higher throughput than Storm Spark Streaming: 670k records/second/node Storm: 115k records/second/node Apache S4: 7.5k records/second/node * Fast Fault Recovery Recovers from faults/stragglers within 1 sec * Real Applications: Conviva Real-time monitoring of video metadata * Achieved 1-2 second latency Millions of video sessions processed Scales linearly with cluster size Real Applications: Mobi