MapReduce优化技术综述.pdfVIP

黄 山，王波涛，王国仁，等.MapReduce优化技术综述[J].计算机科学与探索，2013 ，7（10）：865-885. ISSN 1673-9418 CODEN JKYTA8 E-mail: fcst@vip. 163.com Journal of Frontiers of Computer Science and Technology 1673-9418/2013/07(10)-0865-21 Tel: +86-10 doi: 10.3778/j.issn. 1673-9418.1307035 MapReduce 优化技术综述* + 黄 山，王波涛 ，王国仁，于 戈，李佳佳 东北大学 信息科学与工程学院，沈阳 110819 A Survey on MapReduce Optimization Technologies + HUANG Shan, WANG Botao , WANG Guoren, YU Ge, LI Jiajia College of Information Science and Engineering, Northeastern University, Shenyang 110819, China + Corresponding author: E-mail: wangbotao@ HUANG Shan, WANG Botao, WANG Guoren, et al. A survey on MapReduce optimization technologies. Journal of Frontiers of Computer Science and Technology, 2013, 7(10) ：865-885. Abstract: As a parallel programming model for big data processing, MapReduce is getting more and more attrac- tions from academia and industry for its good scalability, availability and fault tolerance. There exist a lot of optimi- zation technologies focusing on the application limitations of MapReduce. This paper firstly introduces the MapReduce framework, then compares the research work on MapReduce optimization technologies including column storage, index, join, iteration calculation, scientific calculation, and scheduling algorithms respectively. Finally, this paper analyzes the challenges and figures out the trends of this area. Key words: MapReduce; column storage; index; join; iteration; scientific calculation; scheduling algorithms; optimization 摘 要：作为一种处理大数据的并行编程模型，MapReduce 由于其良好的可扩展性、可用性、容错性，得到了学 术界和工业界的关注。针对MapReduce 在应用领域中的不足，已经存在大量的