大数据及OLAP系统.pdfVIP

TOPIC 专题 大数据与OLAP系统 杜小勇，陈跃国，覃雄派 中国人民大学信息学院数据工程与知识工程教育部重点实验室 北京 100872 摘要 数据时代，人们迫切希望在由普通机器组成的大规模集群上能实现高性能的OLAP，然而系统性能的挑战巨 系统性能不断提升。在综述OLAP技术发展的基础上，重点对几个有代表性的SQL on Hadoop系统进行了测 试分析，并展示了这类系统的性能特点。可以预见，未来在低成本的大数据OLAP市场，这类系统会占有重要 位置。 关键词 大数据；OLAP；SQL分析；SQL on Hadoop Big Data and OLAP Systems Du Xiaoyong, Chen Yueguo, Qin Xiongpai Key Laboratory of Data Engineering and Knowledge Engineering, School of Information, Renmin University of China, Beijing 100872, China Abstract OLAP (online analytical processing) is a key technology of business intelligence based on relational data. In big data era, people want to achieve high performance OLAP using a large cluster of ordinary nodes. However, the performance of such systems is a big challenge. Recently, many SQL on Hadoop systems have been proposed to address this challenge. We have seen a significant performance improvement of such systems. A survey of technology development of OLAP technologies was first provided. Then, a study of the performance of three representatives SQL on Hadoop systems was focused on. Based on the results, it is expected that such systems will play an very important role in the market of low cost OLAP analysis. Key words big data, online analytical processing, SQL analysis, SQL on Hadoop 2015005-1 BIG DATA RESEARCH 大数据 大、任务复杂的情况下往往是离线分析任 1 OLAP的前世今生 务。 本文重点围绕OLAP，探讨大数据对 数 据 分析 一 般 指为了从 数 据中获得 于OLAP系统的深刻影响。截至目前，它还 有价值的信息，而采用的诸如数据清理、 是商业数据分析（数据仓库技术）所依赖 建模、查 询、统计、挖掘、展示等操作过 的最为重要也最为核心的数据分析技术。 程，其产生的结果往往用于决策支持，是 使 用 S Q L 技 术 对