基于MapReduce并行编程模型的XBRL维度数据解析算法.docx

文章编号: 2095-6134( 2014) 01-0124-06 基于 Map / Ｒeduce 并行编程模型的 XBＲL 维度数据解析算法* 朱健鹏 ，王 颖，杨 诚 ( 中国科学院大学工程管理与信息技术学院，北京 100049) ( 2013 年 4 月 26 日收稿; 2013 年 5 月 20 日收修改稿) Zhu J P，Wang Y，Yang C． An XBＲL dimensional data parsing algorithm based on the Map / Ｒeduce parallel programming model［J］． Journal of University of Chinese Academy of Sciences，2014，31( 1) : 124-129． 摘 要 从 XBＲL 维度数据处理的角度，研究大规模半结构化数据处理技术，提出一种基于 Map / Ｒeduce 并行编程模型的 XBＲL 维度数据解析算法． 该算法在 Map / Ｒeduce 编程模型和 StAX 流式解析技术的基础上，针对 XBＲL 财务报告中各 XML 文件之间较复杂的数据引用关 系，以整份 XBＲL 财务报告为处理的最小单位，结合并行技术提取维度事项所包含的数据，再 处理业务语义数据，从而实现复杂 XBＲL 维度数据的解析． 性能比较分析表明，该算法在大规 模 XBＲL 数据处理方面具有显著优势． 关键词 XBＲL; 半结构化数据处理; 大数据处理; Map / Ｒeduce; XBＲL 维度 中图分类号: TP399 文献标志码: A doi: 10. 7523 / j． issn． 2095-6134． 2014． 01． 018 An XBＲL dimensional data parsing algorithm based on the Map / Ｒeduce parallel programming model ZHU Jianpeng，WANG Ying，YANG Cheng ( College of Engineering and Information Technology，University of Chinese Academy of Sciences，Beijing 100049，China) Abstract This article intends to study mass semi-structured data processing technology from XBＲL dimensional data processing perspective． A new XBＲL dimensional data parsing algorithm is proposed based on the Map / Ｒeduce parallel programming model and StAX stream parsing technique． The algorithm specifically targets the analysis of complex data reference relationships among XML files in the XBＲL financial report． In order to parse complex XBＲL dimensional data， the algorithm uses a single XBＲL financial report as the minimum processing unit． First，the data are extracted from the dimensional fact items，and then the business semantic data are processed． In experimental tests，the proposed algorithm presents an obvious advantage in large-scale XBＲL data processing． Key words XBＲL; semi-structured data processing; big data processing; Map / Ｒeduce; XBＲL dimension * 国家自然科学基金( 613031