Matei Zaharia：Spark的现状和未来.pdf

The State of Spark And Where We’re Going Next Matei Zaharia Community Growth Project History Spark started as research project in 2009 Open sourced in 2010 st »  1 version was 1600 LOC, could run Wikipedia demo Growing community since Entered Apache Incubator in June 2013 Development Community With over 100 developers and 25 companies, one of the most active communities in big data Comparison: Storm (48), Giraph (52), Drill (18), Tez (12) Past 6 months: more active devs than Hadoop MapReduce! Development Community Healthy across the whole ecosystem Release Growth Spark 0.8: - YARN, MLlib, monitoring UI - 67 contributors Spark 0.7: -  Python API, Spark 0.6: Spark Streaming - Java API, Maven, - 31 contributors standalone mode -  17 contributors Oct ‘12 Feb ‘13 Sept ‘13 Some Community Contributions YARN support (Yahoo!) Columnar compression in Shark (Yahoo!) Fair scheduling (Intel) Metrics reporting (Intel, Quantifind) New RDD operators (Bizo, ClearStory) Scala 2.10 support (Imaginea) Conferences 500 450 400 s 350 e e 300 d n 250 e t t 200 A 150 100 50 0 AMP Camp 1 AMP Camp 2 Spark Summit (Aug 2012) (Aug 2013) (Nov 2013) Projects Built on Spark BlinkDB Spark MLbase GraphX Shark Streaming