基于HBase的空间矢量数据存储模型设计与优化

doi:10.11947/j.AGCS.2020.20190525

测绘学报 ›› 2020, Vol. 49 ›› Issue (10): 1365-1373.doi: 10.11947/j.AGCS.2020.20190525

基于HBase的空间矢量数据存储模型设计与优化

谢鹏^1,2,3, 杨春成⁴, 熊顺^2,3, 何列松^2,3, 周校东^2,3

1. 信息工程大学地理空间信息学院, 河南郑州 450052;
2. 西安测绘研究所, 陕西西安 710054;
3. 地理信息工程国家重点实验室, 陕西西安 710054;
4. 中国地质大学(武汉), 湖北武汉 430074

收稿日期:2019-12-20 修回日期:2020-04-11 发布日期:2020-10-31
作者简介:谢鹏(1977-),男,副研究员,研究方向为空间数据库及应用。E-mail:xp952307@sina.com
基金资助:
国家自然科学基金（41301527）

Design and optimization of spatial vector data storage model based on HBase

XIE Peng^1,2,3, YANG Chuncheng⁴, XIONG Shun^2,3, HE Liesong^2,3, ZHOU Xiaodong^2,3

1. Institute of Geographical Spatial Information, Information Engineering University, Zhengzhou 450052, China;
2. Xi'an Research Institute of Surveying and Mapping, Xi'an 710054, China;
3. State Key Laboratory of Geo-Information Engineering, Xi'an 710054, China;
4. China University of Geosciences, Wuhan 430074, China

Received:2019-12-20 Revised:2020-04-11 Published:2020-10-31
Supported by:
The National Natural Science Foundation of China(No. 41301527)

摘要/Abstract

摘要： 数据存储模型是数据库模型的重要组成部分。本文针对目前关系型空间数据库发展过程中遇到的瓶颈问题以及采用HBase管理空间矢量数据存在的问题，在分析关系型空间数据库存储模型的基础上，将关系型数据库存储模式向HBase存储模式转换规则应用到空间矢量数据管理领域，提出了空间矢量数据关系存储模式向HBase存储模式转换方法，设计了一种空间矢量数据HBase存储模型。利用HBase的实体嵌套、反范式化、无模式等特性，对模型进行了优化完善。通过试验验证了在没有辅助索引情况下，本文设计的存储模型查询效率优于目前常用的基于HBase的空间矢量数据存储模型。

关键词: HBase, 关系型数据库, 空间矢量数据, 存储模型, 反范式化

Abstract: Data storage model is an important part of database model. This paper aims at bottleneck problems in developing relational spatial database and using HBase to manage spatial vector data. Based on the analysis of relational spatial database storage model, the rules fortransforming the relational database storage mode into HBase storage mode are applied to spatial vector data management, a method of transforming spatial vector data relational storage schema into HBase storage schema is proposed, and a HBase storage model is designed for spatial vector data. The model is optimized and improved by using HBase's features such as entity nesting, denormalization and schemalessness.The experimental results show that the query efficiency of the storage model designed in this paper is better than that of the HBase-based spatial vector data storage model in the absence of auxiliary indexes.

Key words: HBase, relational database, spatial vector data, storage model, denormalization

中图分类号:

P208

谢鹏, 杨春成, 熊顺, 何列松, 周校东. 基于HBase的空间矢量数据存储模型设计与优化[J]. 测绘学报, 2020, 49(10): 1365-1373.

XIE Peng, YANG Chuncheng, XIONG Shun, HE Liesong, ZHOU Xiaodong. Design and optimization of spatial vector data storage model based on HBase[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(10): 1365-1373.

参考文献

[1] 吴政, 李成名, 武鹏达, 等. Oracle数据库矢栅数据一体化存储与管理[J]. 测绘学报, 2017, 46(5):639-648. WU Zheng, LI Chengming, WU Pengda, et al. Integrated storage and management of vector and raster data based on oracle database[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(5):639-648.
[2] 陈超, 王亮, 闫浩文, 等. 一种基于NoSQL的地图瓦片数据存储技术[J]. 测绘科学, 2013, 38(1):142-143, 159. CHEN Chao, WANG Liang, YAN Haowen, et al. A map tiles data storage technology based on NoSQL[J]. Science of Surveying and Mapping, 2013, 38(1):142-143, 159.
[3] 廖克. 中国地图学发展的回顾与展望[J]. 测绘学报, 2017, 46(10):1517-1525. DOI:10.11947/j.AGCS.2017.20170330. LIAO Ke. Retrospect and prospect of the development of Chinese cartography[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10):1517-1525. DOI:10.11947/j.AGCS.2017.20170330.
[4] 王家耀, 成毅. 论地图学的属性和地图的价值[J]. 测绘学报, 2015, 44(3):237-241. DOI:10.11947/j.AGCS.2015.20140406. WANG Jiayao, CHENG Yi. Discussions on the attributes of cartography and the value of map[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(3):237-241. DOI:10.11947/j.AGCS.2015.20140406.
[5] 龚健雅. 空间数据库管理系统的概念与发展趋势[J]. 测绘科学, 2001, 26(3):4-9. GONG Jianya. Concepts and development of spatial database management[J]. Science of Surveying and Mapping, 2001, 26(3):4-9.
[6] 喻占武, 郑胜, 李忠民, 等. 基于对象存储的海量空间数据存储与管理[J]. 武汉大学学报(信息科学版), 2008, 33(5):528-532. YU Zhanwu, ZHENG Sheng, LI Zhongmin, et al. Massive spatial data storage and management based on object-based storage[J]. Geomatics and Information Science of Wuhan University, 2008, 33(5):528-532.
[7] 毛静. 基于Oracle Spatial的GIS数据存储管理系统的研究[D]. 西安:长安大学, 2012. MAO Jing. The study of the GIS data storage and management system based on Oracle Spatial[D]. Xi'an:Chang'an University, 2012.
[8] 马元龙. 基于Arc SDE和SQL Server空间数据库的设计与优化方法研究[D]. 合肥:安徽农业大学, 2016. MA Yuanlong. Research on design and optimization methods of spatial database based on Arc SDE and SQL Server[D]. Hefei:Anhui Agricultural University, 2016.
[9] 李剑峰. 三峡水库温室气体空间数据库管理系统设计与开发研究[D]. 武汉:华中科技大学, 2016. LI Jianfeng. Design and development of spatial database management system for the greenhouse gas in Three gorges Reservoir[D]. Wuhan:Huazhong University of Science & Technology, 2016.
[10] SHEKHAR S, CHAWLA S. Spatial databases:a tour[M]. New Jersey:Prentice Hall, Inc.,2003.
[11] GEORGE L. HBase:the definitive guide[M]. Sebastopol:O'Reilly Media, Inc.,2011.
[12] CATTELL R. Scalable SQL and NoSQL data stores[J]. ACM SIGMOD Record, 2011, 39(4):12-27.
[13] 范建永, 龙明, 熊伟. 基于HBase的矢量空间数据分布式存储研究[J]. 地理与地理信息科学, 2012, 28(5):39-42. FAN Jianyong, LONG Ming, XIONG Wei. Research of vector spatial data distributed storage based on HBase[J]. Geography and Geo-Information Science, 2012, 28(5):39-42.
[14] 丁琛. 基于HBase的空间数据分布式存储和并行查询算法研究[D]. 南京:南京师范大学, 2014. DING Chen. Research on distributed storage and parallel query algorithm of spatial data in HBase[D]. Nanjing:Nanjing Normal University, 2014.
[15] GAO Fan, YUE Peng, WUZhaoyan, et al. Geospatial data storage based on HBase and MapReduce[C]//Proceedings of 2017 6th International Conference on Agro-Geoinformatics. Fairfax, VA:IEEE, 2017:1-4.
[16] ZHENG Kun, FU Yanli. Research on vector spatial data storage schema based on Hadoop platform[J]. International Journal of Database Theory and Application, 2013, 6(5):85-94.
[17] LI Chongxin. Transforming relational database into HBase:a case study[C]//Proceedings of 2010 IEEE International Conference on Software Engineering and Service Sciences. Beijing, China:IEEE, 2010:683-687.
[18] 张巍, 龚健雅. 地理信息系统中复杂地物的OO模型[J]. 测绘学报, 1995, 24(4):293-300. ZHANG Wei, GONG Jianya. Object-oriented data model for complex feature in GIS[J]. Acta Geodaetica et Cartographica Sinica, 1995, 24(4):293-300.
[19] 涂振发. 云计算环境下海量空间数据高效存储关键技术研究[D]. 武汉:武汉大学, 2012. TU Zhengfa. Research on the technology of efficient mass spatial data storage in the cloud computing environment[D]. Wuhan:Wuhan University, 2012.
[20] 王家耀. 空间信息系统原理[M]. 北京:科学出版社, 2001. WANG Jiayao. Principles of spatial information system[M]. Beijing:Science Press, 2001.
[21] 王家耀, 孙群, 王光霞, 等. 地图学原理与方法[M]. 2版. 北京:科学出版社, 2014. WANG Jiayao, SUN Qun, WANG Guangxia, et al. Principles and methods of cartography[M]. 2nd ed. Beijing:Science Press, 2014.
[22] 刘仁义, 刘南, 苏国中. 图形数据与关系数据库的结合及其应用[J]. 测绘学报, 2000, 29(4):329-333. DOI:10.3321/j.issn:1001-1595.2000.04.009. LIU Renyi, LIU Nan, SU Guozhong. Manipulation of spatial graphical data in RDBMS and its implementation in application of GIS[J]. Acta Geodaetica et Cartographica Sinica, 2000, 29(4):329-333. DOI:10.3321/j.issn:1001-1595.2000.04.009.
[23] CHANG K T. Introduction to geographic information systems[M]. 8th ed. New York:McGraw-Hill Education, 2016.
[24] 龚健雅. GIS中面向对象时空数据模型[J]. 测绘学报, 1997, 26(4):289-298. GONG Jianya. An object-oriented spatio-temporal data model in GIS[J]. Acta Geodaetica et Cartographica Sinica, 1997, 26(4):289-298.
[25] GAO Fan, YUE Peng, WU Zhaoyan, et al. Geospatial data storage based on HBase and Map Reduce[C]//Proceedings of 2017 6th International Conference on Agro-Geoinformatics. Fairfax, VA, USA:IEEE, 2017:1-4.
[26] HUANG Shidong, CAI Lizhi, LIU Zhenyu, et al. Non-structure data storage technology:a discussion[C]//Proceedings of 2012 IEEE/ACIS 11th International Conference on Computer and Information Science. Shanghai, China:IEEE, 2012:482-487.
[27] 毋河海. 地图数据库系统[M]. 北京:测绘出版社, 1991. WU Hehai. Cartographic database system[M]. Beijing:SinoMaps Press, 1991.
[28] 马延辉, 孟鑫, 李立松. HBase企业应用开发实战[M]. 北京:机械工业出版社, 2014. MA Yanhui, MENG Xin, LI Lisong. Enterprise application development with HBase[M]. Beijing:China Machine Press, 2014.
[29] DIMIDUK N, KHURANA A. HBase in action[M]. New York, USA:Manning Publications, 2012.
[30] 李青云, 余文. 关系型数据库到HBase的转换设计[J]. 信息网络安全, 2015, 15(1):51-55. LI Qinyun, YU Wen. Transition design of relational database into HBase[J]. Netinfo Security, 2015, 15(1):51-55.

基于HBase的空间矢量数据存储模型设计与优化

Design and optimization of spatial vector data storage model based on HBase

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	邱越, 武芳, 翟仁健, 钱海忠, 黄哲琨, 李博. 面向匹配优化的多源建筑物实体级保形空间对齐模型[J]. 测绘学报, 2025, 54(12): 2262-2275.
[2]	张锦彬, 朱军, 党沛, 周宇轩, 杨博文. 现场直播式地理信息服务：基于VR全景的现场实况远程临浸感知[J]. 测绘学报, 2025, 54(12): 2276-2286.
[3]	张岩. 基于街景影像的城市功能区多尺度时空感知方法[J]. 测绘学报, 2025, 54(12): 2289-2289.
[4]	曾进. 城市社会空间的空间大数据量化表达与分析方法：以深圳市为例[J]. 测绘学报, 2025, 54(12): 2292-2292.
[5]	刘少俊. 基于手机信令数据的城市人群活动时空格局分析研究[J]. 测绘学报, 2025, 54(12): 2295-2295.
[6]	吴超, 梁咏翔, 岳瀚, 崔远政, 黄波. 面向计数数据的时空地理加权泊松回归模型[J]. 测绘学报, 2025, 54(11): 2026-2039.
[7]	王小龙, 王卓, 李精忠, 闫浩文. 微地图制图的空间方向关系转译法[J]. 测绘学报, 2025, 54(11): 2040-2051.
[8]	胡鑫, 杨学习, 江一凡, 王宪彬, 丁晨, 谢顾然, 邓敏. 基于多智能体层次化协同的地理事件抽取与时空解析[J]. 测绘学报, 2025, 54(11): 2052-2067.
[9]	李俊, 李朝奎, 黄磊, 冯媛媛. 高速公路广告牌巡检目标跟踪的改进ByteTrack算法[J]. 测绘学报, 2025, 54(11): 2068-2080.
[10]	叶欣宇, 徐胜华, 刘纪平, 陈虹宇, 王琢璐, 李维炼. 基于时空因果推断的下一个兴趣点推荐[J]. 测绘学报, 2025, 54(11): 2081-2096.
[11]	赵学胜, 谢文澜, 孙文彬. 空间格网互操作的研究进展与关键问题[J]. 测绘学报, 2025, 54(10): 1727-1740.
[12]	高凡, 路威, 甘麟露, 章繁, 荣凤娟, 汤士涵. 智能驱动的并行地理计算引擎框架[J]. 测绘学报, 2025, 54(10): 1877-1892.
[13]	吴浩宇, 朱庆, 丁雨淋, 鲍榴, 刘利. 数据模型知识协同驱动的隧道围岩高精度数字孪生建模方法[J]. 测绘学报, 2025, 54(10): 1893-1906.
[14]	郝彧露. 时空数据驱动的城市区域火灾风险评估预测模型及应用[J]. 测绘学报, 2025, 54(10): 1910-1910.
[15]	张付兵, 孙群, 徐青, 马京振, 黄文君, 陈若虚. 随机森林和图神经网络支持下的河系自动分级与选取方法[J]. 测绘学报, 2025, 54(9): 1697-1711.