面向高性能计算机体系结构的MapReduce编程框架优化研究-计算机科学与技术专业毕业论文.docxVIP

国防科 国防科学技术大学研究生院工学硕士学位论文 第 I 第 I 页 万方数据 目 录 摘 要 i ABSTRACT ii 第一章 绪论········································································································ 1 1.1 研究背景 ··········································································································· 1 1.1.1 大数据时代的来临 ················································································ 1 1.1.2 高性能计算机系统处理大规模数据的现状及问题 ···························· 2 1.1.3 MapReduce 技术处理大规模数据的现状及问题 3 1.2 研究意义和难点 ······························································································· 3 1.2.1 研究意义 ································································································ 4 1.2.2 研究难点 ································································································ 5 1.3 本文工作和章节安排 ······················································································· 5 1.3.1 本文的工作内容 ···················································································· 5 1.3.1 本文的组织 ···························································································· 6 第二章 相关技术介绍 ························································································· 7 2.1 高性能计算机系统及传统并行编程模型 ······················································· 7 2.1.1 高性能计算机系统的发展现状与趋势 ················································ 7 2.1.2 传统并行编程模型的现状及特点 ························································ 8 MapReduce 编程模型 10 MapReduce 的来源与背景 10 MapReduce 的编程模式 10 Google 的 MapReduce 实现 11 MapReduce 的开源实现 Hadoop 12 2.3 本章小结 ········································································································· 15 第三章 在高性能计算机体系结构下部署 MapReduce 框架 16 3.1 概述 ················································································································· 16 HDFS 与 Lustre 作为 Hadoop 后端存储的特点分析 16 基于 HDFS 的 Hadoop 处理平台 16 基于 Lustre 的 Hadoop 处