Design and optimization of spatial vector data storage model based on HBase

doi:10.11947/j.AGCS.2020.20190525

Abstract

Abstract: Data storage model is an important part of database model. This paper aims at bottleneck problems in developing relational spatial database and using HBase to manage spatial vector data. Based on the analysis of relational spatial database storage model, the rules fortransforming the relational database storage mode into HBase storage mode are applied to spatial vector data management, a method of transforming spatial vector data relational storage schema into HBase storage schema is proposed, and a HBase storage model is designed for spatial vector data. The model is optimized and improved by using HBase's features such as entity nesting, denormalization and schemalessness.The experimental results show that the query efficiency of the storage model designed in this paper is better than that of the HBase-based spatial vector data storage model in the absence of auxiliary indexes.

Key words: HBase, relational database, spatial vector data, storage model, denormalization

CLC Number:

P208

XIE Peng, YANG Chuncheng, XIONG Shun, HE Liesong, ZHOU Xiaodong. Design and optimization of spatial vector data storage model based on HBase[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(10): 1365-1373.

References

[1] 吴政, 李成名, 武鹏达, 等. Oracle数据库矢栅数据一体化存储与管理[J]. 测绘学报, 2017, 46(5):639-648. WU Zheng, LI Chengming, WU Pengda, et al. Integrated storage and management of vector and raster data based on oracle database[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(5):639-648.
[2] 陈超, 王亮, 闫浩文, 等. 一种基于NoSQL的地图瓦片数据存储技术[J]. 测绘科学, 2013, 38(1):142-143, 159. CHEN Chao, WANG Liang, YAN Haowen, et al. A map tiles data storage technology based on NoSQL[J]. Science of Surveying and Mapping, 2013, 38(1):142-143, 159.
[3] 廖克. 中国地图学发展的回顾与展望[J]. 测绘学报, 2017, 46(10):1517-1525. DOI:10.11947/j.AGCS.2017.20170330. LIAO Ke. Retrospect and prospect of the development of Chinese cartography[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10):1517-1525. DOI:10.11947/j.AGCS.2017.20170330.
[4] 王家耀, 成毅. 论地图学的属性和地图的价值[J]. 测绘学报, 2015, 44(3):237-241. DOI:10.11947/j.AGCS.2015.20140406. WANG Jiayao, CHENG Yi. Discussions on the attributes of cartography and the value of map[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(3):237-241. DOI:10.11947/j.AGCS.2015.20140406.
[5] 龚健雅. 空间数据库管理系统的概念与发展趋势[J]. 测绘科学, 2001, 26(3):4-9. GONG Jianya. Concepts and development of spatial database management[J]. Science of Surveying and Mapping, 2001, 26(3):4-9.
[6] 喻占武, 郑胜, 李忠民, 等. 基于对象存储的海量空间数据存储与管理[J]. 武汉大学学报(信息科学版), 2008, 33(5):528-532. YU Zhanwu, ZHENG Sheng, LI Zhongmin, et al. Massive spatial data storage and management based on object-based storage[J]. Geomatics and Information Science of Wuhan University, 2008, 33(5):528-532.
[7] 毛静. 基于Oracle Spatial的GIS数据存储管理系统的研究[D]. 西安:长安大学, 2012. MAO Jing. The study of the GIS data storage and management system based on Oracle Spatial[D]. Xi'an:Chang'an University, 2012.
[8] 马元龙. 基于Arc SDE和SQL Server空间数据库的设计与优化方法研究[D]. 合肥:安徽农业大学, 2016. MA Yuanlong. Research on design and optimization methods of spatial database based on Arc SDE and SQL Server[D]. Hefei:Anhui Agricultural University, 2016.
[9] 李剑峰. 三峡水库温室气体空间数据库管理系统设计与开发研究[D]. 武汉:华中科技大学, 2016. LI Jianfeng. Design and development of spatial database management system for the greenhouse gas in Three gorges Reservoir[D]. Wuhan:Huazhong University of Science & Technology, 2016.
[10] SHEKHAR S, CHAWLA S. Spatial databases:a tour[M]. New Jersey:Prentice Hall, Inc.,2003.
[11] GEORGE L. HBase:the definitive guide[M]. Sebastopol:O'Reilly Media, Inc.,2011.
[12] CATTELL R. Scalable SQL and NoSQL data stores[J]. ACM SIGMOD Record, 2011, 39(4):12-27.
[13] 范建永, 龙明, 熊伟. 基于HBase的矢量空间数据分布式存储研究[J]. 地理与地理信息科学, 2012, 28(5):39-42. FAN Jianyong, LONG Ming, XIONG Wei. Research of vector spatial data distributed storage based on HBase[J]. Geography and Geo-Information Science, 2012, 28(5):39-42.
[14] 丁琛. 基于HBase的空间数据分布式存储和并行查询算法研究[D]. 南京:南京师范大学, 2014. DING Chen. Research on distributed storage and parallel query algorithm of spatial data in HBase[D]. Nanjing:Nanjing Normal University, 2014.
[15] GAO Fan, YUE Peng, WUZhaoyan, et al. Geospatial data storage based on HBase and MapReduce[C]//Proceedings of 2017 6th International Conference on Agro-Geoinformatics. Fairfax, VA:IEEE, 2017:1-4.
[16] ZHENG Kun, FU Yanli. Research on vector spatial data storage schema based on Hadoop platform[J]. International Journal of Database Theory and Application, 2013, 6(5):85-94.
[17] LI Chongxin. Transforming relational database into HBase:a case study[C]//Proceedings of 2010 IEEE International Conference on Software Engineering and Service Sciences. Beijing, China:IEEE, 2010:683-687.
[18] 张巍, 龚健雅. 地理信息系统中复杂地物的OO模型[J]. 测绘学报, 1995, 24(4):293-300. ZHANG Wei, GONG Jianya. Object-oriented data model for complex feature in GIS[J]. Acta Geodaetica et Cartographica Sinica, 1995, 24(4):293-300.
[19] 涂振发. 云计算环境下海量空间数据高效存储关键技术研究[D]. 武汉:武汉大学, 2012. TU Zhengfa. Research on the technology of efficient mass spatial data storage in the cloud computing environment[D]. Wuhan:Wuhan University, 2012.
[20] 王家耀. 空间信息系统原理[M]. 北京:科学出版社, 2001. WANG Jiayao. Principles of spatial information system[M]. Beijing:Science Press, 2001.
[21] 王家耀, 孙群, 王光霞, 等. 地图学原理与方法[M]. 2版. 北京:科学出版社, 2014. WANG Jiayao, SUN Qun, WANG Guangxia, et al. Principles and methods of cartography[M]. 2nd ed. Beijing:Science Press, 2014.
[22] 刘仁义, 刘南, 苏国中. 图形数据与关系数据库的结合及其应用[J]. 测绘学报, 2000, 29(4):329-333. DOI:10.3321/j.issn:1001-1595.2000.04.009. LIU Renyi, LIU Nan, SU Guozhong. Manipulation of spatial graphical data in RDBMS and its implementation in application of GIS[J]. Acta Geodaetica et Cartographica Sinica, 2000, 29(4):329-333. DOI:10.3321/j.issn:1001-1595.2000.04.009.
[23] CHANG K T. Introduction to geographic information systems[M]. 8th ed. New York:McGraw-Hill Education, 2016.
[24] 龚健雅. GIS中面向对象时空数据模型[J]. 测绘学报, 1997, 26(4):289-298. GONG Jianya. An object-oriented spatio-temporal data model in GIS[J]. Acta Geodaetica et Cartographica Sinica, 1997, 26(4):289-298.
[25] GAO Fan, YUE Peng, WU Zhaoyan, et al. Geospatial data storage based on HBase and Map Reduce[C]//Proceedings of 2017 6th International Conference on Agro-Geoinformatics. Fairfax, VA, USA:IEEE, 2017:1-4.
[26] HUANG Shidong, CAI Lizhi, LIU Zhenyu, et al. Non-structure data storage technology:a discussion[C]//Proceedings of 2012 IEEE/ACIS 11th International Conference on Computer and Information Science. Shanghai, China:IEEE, 2012:482-487.
[27] 毋河海. 地图数据库系统[M]. 北京:测绘出版社, 1991. WU Hehai. Cartographic database system[M]. Beijing:SinoMaps Press, 1991.
[28] 马延辉, 孟鑫, 李立松. HBase企业应用开发实战[M]. 北京:机械工业出版社, 2014. MA Yanhui, MENG Xin, LI Lisong. Enterprise application development with HBase[M]. Beijing:China Machine Press, 2014.
[29] DIMIDUK N, KHURANA A. HBase in action[M]. New York, USA:Manning Publications, 2012.
[30] 李青云, 余文. 关系型数据库到HBase的转换设计[J]. 信息网络安全, 2015, 15(1):51-55. LI Qinyun, YU Wen. Transition design of relational database into HBase[J]. Netinfo Security, 2015, 15(1):51-55.