大数据全生命周期中关键问题研究及应用.pdfVIP

第35 卷 第1 期 中 国 电 机 工 程 学 报 Vol.35 No.1 Jan.5, 2015 2015 年1 月5 日 Proceedings of the CSEE ©2015 Chin.Soc.for Elec.Eng. 23 DOI ：10.13334/j.0258-8013.pcsee.2015.01.003 文章编号：0258-8013 (2015) 01-0023-06 中图分类号：TP 311 大数据全生命周期中关键问题研究及应用 1 2 2 3 刘道新 ，胡航海 ，张健 ，徐秀敏 (1．华北电力大学经济与管理学院，北京市 昌平区 102206 ；2．国家电网公司，北京市 西城区 100031 ； 3．北京中电普华信息技术有限公司，北京市 海淀区 100192) Research on Key Issues of Big Data Lifecycle and Its Applications 1 2 2 3 LIU Daoxin , HU Hanghai , ZHANG Jian , XU Xiumin (1. School of Economics and Management, North China Electric Power University, Changping District, Beijing 102206, China; 2. State Grid Corporation of China, Xicheng District, Beijing 100031, China; 3. China Power Information Technology of Beijing, Haidian District, Beijing 100192, China) ABSTRACT: With the big data era arriving, the research on 关键词：大数据；压缩存储；缓存机制；动态配置；统一数 big data lifecycle followed one by one and become research 据资源库 hotspots, such as in obtaining, storage, processing and mining. 0 引言 This paper applied electrical, financial, economical data domestic and overseas as research objects to solve storage, 大数据被认为是无法在一定时间内用常规软 processing and mining problems of big data. In this paper, the 件工具对其内容进行抓取、管理和处理的数据集 categories of big data were analyzed on the basis of its features. 合[1]，其更多意义在于通过对这些数据的交换、整 The big data compressed storage model, which based on 合和分析，可以发现新知识、创造新价值，带来“大 improved sparse matrix, index dimensions and separated index