基于MapReduce的联机分析服务器查询处理分析-计算机软件与理论专业论文.docxVIP

华中科技大学硕士学位论文 华 中 科 技 大 学 硕 士 学 位 论 文 II II Abstract As the Internets rapid development and wide application of information technology, increasingly large amounts of data are produced by the network, and on-line analytical processing, as the primary technology to store and analyze data, is required to make correspondingly the amount of data multiplied stored. At the same time, dealing with huge amounts of data, we need to take a huge amount of computation. MapReduce proposed by Google Inc. is a large computer cluster framework model of concurrent processing huge amounts of data, but has inherent deficiencies in handling structured data. Therefore, research on hybrid system of MapReduce and database has important significance. In this paper, we firstly analyze the design requirements of hybrid system based on MapReduce and database from system demands, design principles and targets; Secondly, we give the overall architecture of the system, and analyze the system from tomography structure and system module, tomography structure includes presentation layer, conversion layer, computation/scheduling layer and storage layer, system module includes distributed database storage optimization model, query optimization model etc., and then descript the main work processes of the system; Finally, we extend a new multidimensional query language, and descript the syntax detail of the multidimensional query language. In terms of optimization for hybrid system technology, we give implementations of storage and query optimization technology. For storage optimization, we define the manner of division and store of fact tables and dimension tables, and the optimization fomula of joining between the dimension table and fact table; for query optimization, we improve the algorithm of structure and query of QCTree, and its implementation on MapReduce; finally we analyze the effect of storage optimization to query efficiency. Finally, the experiments show that the performance of the system o