Continuous indoor visual localization using a perceptual Hash algorithm and spatial constraint

doi:10.11947/j.AGCS.2021.20200286

Abstract

Abstract: Visual localization achieves indoor localization by matching visual data (collected by camera) with visual features of an environment. However, visual feature matching requires a long computation time, which makes it difficult to provide a continuous localization result. Besides, for environment with sparse visual data (e.g. images), it is also difficult to achieve continuous indoor localization using only visual feature matching. To solve this problem, this study proposes a continuous indoor localization approach using perceptual Hash algorithm (pHash) and spatial-constrained image searching strategies. It realizes accurate indoor localization by matching the collected video frames (from smartphone) with the images from a generated indoor image dataset. To improve the efficiency of visual feature matching, a two-level image searching and matching strategy is designed, including a pHash-based global searching strategy and a local strategy considering motion continuity. Based on the two-level strategy, a continuous indoor visual localization algorithm is proposed, which can increase the spatial continuity of localization result by integrating both visual localization and dead reckoning. Besides, this algorithm employs a structure from motion method to improve its heading estimation accuracy. Experimental results show that the localization errors of the image querying, continuous offline localization and online localization of this method are approximately 0.70, 0.86 and 0.93 m, respectively, which achieves sub-meter level localization accuracy. In the online localization condition, its average computation time is about 0.42 s, which can provide continuous visual localization.

Key words: indoor localization, visual localization, image matching, perceptual Hash, structure from motion

CLC Number:

P228

ZHANG Xing, LIN Jing, LI Qingquan, LIU Tao, FANG Zhixiang. Continuous indoor visual localization using a perceptual Hash algorithm and spatial constraint[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(12): 1639-1649.

References

[1] 陈锐志,陈亮.基于智能手机的室内定位技术的发展现状和挑战[J].测绘学报, 2017, 46(10):1316-1326.DOI:10.11947/j.AGCS.2017.20170383. CHEN Ruizhi, CHEN Liang. Indoor positioning with smartphones:the state-of-the-art and the challenges[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10):1316-1326. DOI:10.11947/j.AGCS.2017.20170383.
[2] 李清泉,周宝定,马威,等. GIS辅助的室内定位技术研究进展[J].测绘学报, 2019, 48(12):1498-1506.DOI:10.11947/j.AGCS.2019.20190455. LI Qingquan, ZHOU Baoding, MA Wei, et al. Research process of GIS-aided indoor localization[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(12):1498-1506. DOI:10.11947/j.AGCS.2019.20190455.
[3] LIM C H, WAN Yahong, NG B P, et al. A real-time indoor WiFi localization system utilizing smart antennas[J]. IEEE Transactions on Consumer Electronics, 2007, 53(2):618-622.
[4] 吴东金,夏林元.面向室内WLAN定位的动态自适应模型[J].测绘学报, 2015, 44(12):1322-1330.DOI:10.11947/j.AGCS.2015.20130780. WU Dongjin, XIA Linyuan. Dynamic adaptive model for indoor WLAN localization[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(12):1322-1330. DOI:10.11947/j.AGCS.2015.20130780.
[5] ZHUANG Yuan, YANG Jun, LI You, et al. Smartphone-based indoor localization with bluetooth low energy beacons[J]. Sensors, 2016, 16(5):596-615.
[6] HAZAS M, HOPPER A. Broadband ultrasonic location systems for improved indoor positioning[J]. IEEE Transactions on Mobile Computing, 2006, 5(5):536-547.
[7] ATHALYE A, SAVIC V, BOLIC M, et al. Novel semi-passive RFID system for indoor localization[J]. IEEE Sensors Journal, 2013, 13(2):528-537.
[8] DE ANGELIS A, DWIVEDI S, HÄNDEL P. Characterization of a flexible UWB sensor for indoor localization[J]. IEEE Transactions on Instrumentation and Measurement, 2013, 62(5):905-913.
[9] SUBBU K P, GOZICK B, DANTU R. Locate me:magnetic-fields-based indoor localization using smartphones[J]. ACM Transactions on Intelligent Systems and Technology, 2013, 4(4):73-99.
[10] LOWRY S, SVNDERHAUF N, NEWMAN P, et al. Visual place recognition:a survey[J]. IEEE Transactions on Robotics, 2016, 32(1):1-19.
[11] DI Kaichang, WAN Wenhui, ZHAO Hongying, et al. Progress and applications of visual SLAM[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(2):38-49. DOI:10.11947/j.JGGS.2019.0205.
[12] CHENG Chuanqi, HAO Xiangyang, LI Jiansheng, et al. A robust Gaussian mixture model for mobile robots' vision-based pose estimation[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(3):79-90. DOI:10.11947/j.JGGS.2019.0308.
[13] HUANG J Y, LEE S H, TSAI C H. A fast image matching technique for the panoramic-based localization[C]//Proceedings of 2016 IEEE/ACIS:the 15th International Conference on Computer and Information Science (ICIS).Okayama, Japan:IEEE, 2016:1-6.
[14] SATTLER T, LEIBE B, KOBBELT L. Fast image-based localization using direct 2D-to-3D matching[C]//Proceedings of 2011 International Conference on Computer Vision.Barcelona, Spain:IEEE, 2011:667-674.
[15] GAO Ruipeng, TIAN Yang, YE Fan, et al. Sextant:towards ubiquitous indoor localization service by photo-taking of the environment[J]. IEEE Transactions on Mobile Computing, 2016, 15(2):460-474.
[16] XIAO Aoran, CHEN Ruizhi, LI Deren, et al. An indoor positioning system based on static objects in large indoor scenes by using smartphone cameras[J]. Sensors, 2018, 18(7):2229.
[17] WU Dewen, CHEN Ruizhi, CHEN Liang. Visual positioning indoors:human eyes vs. smartphone cameras[J]. Sensors, 2017, 17(11):2645-2650.
[18] WU Teng, LIU Jingbin, LI Zheng, et al. Accurate smartphone indoor visual positioning based on a high-precision 3D photorealistic map[J]. Sensors, 2018, 18(6):1974-1994.
[19] ZHOU Yan, ZHENG Xianwei, CHEN Ruizhi, et al. Image-based localization aided indoor pedestrian trajectory estimation using smartphones[J]. Sensors, 2018, 18(1):258-276.
[20] LOWE D G. Distinctive image features from scale-invariant key points[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
[21] BAY H, ESS A, TUYTELAARS T, et al. Speeded-up robust features (SURF)[J]. Computer Vision and Image Understanding, 2008, 110(3):346-359.
[22] SIAGIAN C, ITTI L. Rapid biologically-inspired scene classification using features shared with visual attention[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(2):300-312.
[23] LIU Tao, ZHANG Xing, LI Qingquan, et al. A visual-based approach for indoor radio map construction using smartphones[J]. Sensor,2017, 17(8):1790-1807.
[24] LIU Tao, ZHANG Xing, LI Qingquan, et al. An accurate visual-inertial integrated geo-tagging method for crowdsourcing-based indoor localization[J]. Remote Sensing, 2019, 11(16):1912-1934.
[25] SCHNEIDER M, CHANG S F. A robust content based digital signature for image authentication[C]//Proceedings of the 3rd IEEE International Conference on Image Processing.Lausanne, Switzerland:IEEE, 1996, 3:227-230.
[26] FRIDRICH J, GOLJAN M. Robust hash functions for digital watermarking[C]//Proceedings of 2000 International Conference on Information Technology:Coding and Computing.Las Vegas, NV, USA:IEEE, 2000:178-183.
[27] 乔君.基于图像内容安全的感知哈希方法研究[D].上海:上海应用技术大学, 2017. QIAO Jun. Research on perceptual Hashing based on image content security[D]. Shanghai:Shanghai Institute of Technology, 2017.
[28] 陈斌.基于感知哈希算法的监控视频认证研究[D].深圳:深圳大学, 2017. CHEN Bin. Research on surveillance video authentication based on perceptual Hash algorithm[D]. Shenzhen:Shenzhen University, 2017.
[29] 王前.基于视频感知哈希的动态手势跟踪方法研究[D].兰州:兰州理工大学, 2018. WANG Qian. Research on dynamic hand gesture tracking method based on video perceptual hashing[D]. Lanzhou:Lanzhou University of Technology, 2018.
[30] 陈优良,肖钢,卞焕,等.一种融合动态预测的感知哈希目标跟踪算法[J].测绘通报, 2020(2):17-23.DOI:10.13474/j.cnki.11-2246.2020.0038. CHEN Youliang, XIAO Gang, BIAN Huan, et al. A perceptual Hash target tracking algorithm based on dynamic prediction[J]. Bulletin of Surveying and Mapping, 2020(2):17-23. DOI:10.13474/j.cnki.11-2246.2020.0038.
[31] 丁凯孟.基于感知哈希的遥感影像认证算法研究[J].测绘学报, 2017, 46(9):1205-1205.DOI:10.11947/j.AGCS.2017.20170233. DING Kaimeng. Perceptual Hashing based authentication algorithm research for remote sensing image[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(9):1205-1205. DOI:10.11947/j.AGCS.2017.20170233.
[32] 张洪帅.基于图像感知哈希的场景分类[D].兰州:兰州大学, 2018. ZHANG Hongshuai. Scene classification based on image perception Hashing[D]. Lanzhou:Lanzhou University, 2018.
[33] LOWE D G. Object recognition from local scale-invariant features[C]//Proceedings of the 7th International Conference on Computer Vision.Kerkyra, Greece:IEEE, 1999, 2:1150-1157.
[34] FISCHLER M A, BOLLES R C. Random sample consensus:A paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6):381-395.