大数据原理与应用_论文.docVIP

摘要 随着计算机技术以及互联网运用高速的扩展到人类社会生产生活的各个方面数据量呈现出爆发性的增长。如今大数据集以及超大数据集的存储和处理已成为很多企业面临的新的挑战。 大数据正在改变着产品和生产过程、企业和产业，甚至竞争本身的性质。把信息技术看作是辅助或服务性的工具已经成为过时的观念，管理者应该认识到信息技术的广泛影响和深刻含义，以及怎样利用信息技术来创造有力而持久的竞争优势。无疑，信息技术正在改变着我们习以为常的经营之道，一场关系到企业生死存亡的技术革命已经到来。 概述了概念意义及发展前景，介绍了大数据关键技术和开发工具。依据官方对式系统架构adoop及其重要组件apReduce编程模型了阐述并大数据一体机概念进行了解列举了一些大数据应用方面的分析。 关键词 Hadoop MapReduce 大数据应用 ABSTRACT With the development of computer technology and the Internet with high speed is extended to the production and life of human society in all aspects, the amount of data showing explosive growth. Now, big data and storage and processing of large data has become a new challenge for many enterprises. Big data is changing the products and processes, enterprises and industries, and even the nature of competition itself. Taking auxiliary information technology as a tool or service has become outdated concept, Managers should be aware of the extensive influence of the information technology and profound meaning, and know how to use information technology to create strong and lasting competitive advantage. Undoubtedly, information technology is changing the way we were accustomed to, a vital important technology revolution of enterprise has come. This paper firstly introduces the concept significance and development prospect of big data, then according to official site focuses on the distributed system architecture of Hadoop and its important components of MapReduce programming model, and get the basic understanding of the concept of big data machine. Finally, lists some analysis case of big data applications. 目录 摘要 I ABSTRACT II 目录 III 一、 绪论 1 1. 大数据发展背景及意义 1 2. 研究现状 1 3. 未来展望 3 4. 大数据基础与技术盘点 4 4.1大数据概念 4 4.2大数据用途意义及弊端 4 4.3大数据技术盘点 5 4.3.1分布式系统基础架构Hadoop 5 4.3.2编程模型MapReduce 5 4.3.3分布式数据库HBase 5 二、 Hadoop框架的工作机制 6 1. 前提设计和目标 6 1.1硬件错误 6 1.2流式数据访问 6 1.3大规模数据集 6 1.4简单的一致性模型 6 1.5“移动计算比移动数据更划算” 7 1.6异构软硬件平台间的可移植性 7 2. Namenode和Datanode 7 3. 文件系统的名字空间 (namespac