基于云技术的智慧城中大规模房屋数据处理-计算机系统应用.pdfVIP

计 算 机 系 统 应 用 2014 年 第 23 卷 第 12 期 基于云技术的智慧城市中大规模房屋数据处理① 1 2 曾羽琚 , 陈明辉 1(湖南大学长沙环境保护职业技术学院 环境信息系, 长沙 410004) 2(湖南商学院 教务处, 长沙 410205) 摘 要: 在智慧城市中传感器和设备收集各种数据, 实现增值服务. 主要关注来自智慧城市中的智能房屋数据, 提出用于存储和处理大规模房屋数据的Scallop4SC 平台, 实现智慧城市服务. 房屋数据分为房屋日志数据和房屋 配置数据, 分别使用 HBase 和 MYSQL 管理. 通过 HBase 键值以 schemaless 方式存储管理异构日志数据. MySQL 管理配置数据, 完成房屋数据查询. 多节点 Hadoop/MapReduce 集群处理巨量房屋日志数据. 最后通过实际一年 的房屋日志数据, 计算智能设备能耗并完成实验性评估. 根据实验结果, 探讨 Scallop4SC 处理智慧城市的数据规 模. 关键词: 智慧城市; 智能住宅; 数据平台; Hadoop; HBase Using Cloud Technologies for Large-Scale House Data in Smart City 1 2 ZENG Yu-Ju , CHEN Ming-Hui 1(Changsha Environmental Protection College, Hunan University, Changsha 410005, China) 2(Hunan University of Commerce, Changsha 410205, China) Abstract: In the smart city environment, a wide variety of data are collected from sensors and devices to achieve value-added services. In this paper, we especially focus on data taken from smart houses in the smart city, and propose a platform, called Scallop4SC, that stores and processes the large-scale house data. The house data is classified into log data or configuration data. Since the amount of the log is extremely large, we introduce the Hadoop/MapReduce with a multi-node cluster. On top of this, we use HBase key-value store to manage heterogeneous log data in a schemaless manner. On the other hand, to manage the configuration data, we choose MySQL to process various queries to the house data efficiently. We propose practical data models of the log data and the configuration data on HBase and MySQL, respectively. Then we show how Scallop4SC works as an efficient data platform for smart city services. We conduct an experimental evaluation t