基于HBase海量GIS数据分布式处理实践.PDFVIP

TOPIC 专题 73 基于HBase的海量GIS数据 分布式处理实践 1 1 1 1,2 1,2 李雪梅 ，邢俊峰，刘大伟，王海洋 ，刘玮 1.烟台中科网络技术研究所，山东 烟台 264003；2.中国科学院计算技术研究所，北京 100080 摘要 设计了一种基于分布式数据库HBase的GIS数据管理系统。系统优化了栅格数据的生成和存储过程，将海量 栅格数据直接写入HBase存储、索引。同时，针对矢量空间数据的存储、索引与检索，提出了一种新的rowkey 设计，既考虑经纬度，又考虑空间数据类型和属性，使得在按空间位置检索矢量地理信息时，能通过HBase的 rowkey迅速定位需要返回的数据。在HBase的集群环境上用真实GIS数据对上述方法进行了验证，结果表明， 提出的系统具有较高的海量数据存储和检索性能，实现了海量地理信息数据的高效存储和实时高速检索。 关键词 大数据；HBase；栅格数据；矢量数据；rowkey doi: 10.11959/j.issn.2096-0271.2016032 Distributed processing practice of the massive GIS data based on HBase 1 1 1 1,2 1,2 LI Xuemei , XING Junfeng , LIU Dawei , WANG Haiyang , LIU Wei 1. Institute of Network Technology, ICT(YANTAI), Yantai 264003, China 2. Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100080, China Abstract Based on the distributed database HBase, a kind of GIS data management system was designed. The system optimized the generated and stored procedures of raster data, which could be directly written into the storage and indexing of the HBase. At the same time, in view of the storing, indexing and retrieval of the vector spatial data, a new design for rowkey was proposed that considering both the latitude and longitude, and the spatial data types and attributes. So that the data needed to be returned could be quickly located by rowkey of the HBase, when retrieving vector geographic information according to the spatial location. The above methods had been verified on the HBase cluster environment with real GIS data. The results show that the proposed system has high performance for storage and retrieval of mass data, and realizes the efficient storage and real-tim