《vldb11-job-optimization》.pdf

Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs Herodotos Herodotou Shivnath Babu Duke University Duke University hero@ shivnath@ ABSTRACT A MapReduce program expresses a computation over input data through two functions: map and reduce . MapReduce has emerged as a viable competitor to database sys- The map function is invoked for every key-value pair tems in big data analytics. MapReduce programs are being written for a wide variety of application domains including business data in the input data to output zero or more key-value pairs of the form . The reduce function is invoked for ev- processing, text analysis, natural language processing, Web graph ery unique key and corresponding values in the map and social network analysis, and computational science. However, output. red