Review of visual SLAM environment perception technology and intelligent surveying and mapping application

doi:10.11947/j.AGCS.2023.20220240

Abstract

Abstract: Visual SLAM(simultaneous localization and mapping) technology is one of the core technologies of modern intelligent equipment environment perception. It is an important factor driving the development of modern surveying and mapping production mode to intelligent surveying and mapping. Focusing on the visual SLAM environment perception technology, this paper combs the brief technical framework, important algorithms and mapping application mode of typical visual SLAM environment perception in the past 30 years according to the five aspects of feature method, direct method, visual fingerprint database, semantic SLAM and brain-like SLAM; summarizes and analyzes the development trend of visual SLAM environment perception technology in intelligent environment interaction perception, crowdsourcing instant information processing, and diversified perception data service; discusses the application mode of visual SLAM environment perception technology in interactive navigation and positioning, digital twin city construction, real-time surface monitoring and interpretation, crowdsourcing map POI production, unattended geological disaster monitoring and deep space exploration supported by independent interactive ability. At present, the surveying and mapping industry is in a period of major development and transformation. With the rapid combination of visual SLAM environment perception technology and artificial intelligence technology, it will further enable the transformation of surveying and mapping production mode and improve the level of intelligent production and service.

Key words: visual SLAM, SLAM technical framework, environment perception, intelligent surveying and mapping

CLC Number:

P237

ZHANG Jixian, LIU Fei. Review of visual SLAM environment perception technology and intelligent surveying and mapping application[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(10): 1617-1630.

References

[1] 杨元喜,李晓燕.微PNT与综合PNT[J].测绘学报, 2017, 46(10):1249-1254. DOI:10.11947/j.AGCS.2017. 20170249. YANG Yuanxi, LI Xiaoyan. Micro-PNT and comprehensive PNT[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10):1249-1254. DOI:10.11947/j.AGCS.2017. 20170249.
[2] 刘经南,郭文飞,郭迟,等.智能时代泛在测绘的再思考[J].测绘学报, 2020, 49(4):403-414. DOI:10.11947/j.AGCS.2020. 20190539. LIU Jingnan, GUO Wenfei, GUO Chi, et al. Rethinking ubiquitous mapping in the intelligent age[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(4):403-414. DOI:10.11947/j.AGCS.2020. 20190539.
[3] MUR-ARTAL R, TARDÓS J D. ORB-SLAM2:an open-source SLAM system for monocular, stereo, and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017, 33(5):1255-1262.
[4] DI Kaichang, LIU Zhaoqin, WAN Wenhui, et al. Geospatial technologies for Chang'e-3 and Chang'e-4 lunar rover missions[J]. Geo-Spatial Information Science, 2020, 23(1):87-97.
[5] 李德仁.论时空大数据的智能处理与服务[J].地球信息科学学报, 2019, 21(12):1825-1831. LI Deren. The intelligent processing and service of spatiotemporal big data[J]. Journal of Geo-Information Science, 2019, 21(12):1825-1831.
[6] WANG Sufang, WU Zheng, ZHANG Weicun. An overview of SLAM[C]//Proceedings of 2018 Chinese Intelligent Systems Conference. Singapore:Springer, 2018:673-681.
[7] 高翔,张涛,刘毅.视觉SLAM十四讲:从理论到实践[M].北京:电子工业出版社, 2017. GAO Xiang, ZHANG Tao, LIU Yi. Fourteen lectures on visual SLAM:from theory to practice[M]. Beijing:Publishing House of Electronics Industry, 2017.
[8] DURRANT-WHYTE H, BAILEY T. Simultaneous localization and mapping:part I[J]. IEEE Robotics&Automation Magazine, 2006, 13(2):99-110.
[9] BAILEY T, DURRANT-WHYTE H. Simultaneous localization and mapping (SLAM):part II[J]. IEEE Robotics&Automation Magazine, 2006, 13(3):108-117.
[10] CADENA C, CARLONE L, CARRILLO H, et al. Past, present, and future of simultaneous localization and mapping:toward the robust-perception age[J]. IEEE Transactions on Robotics, 2016, 32(6):1309-1332.
[11] 龚健雅.人工智能时代测绘遥感技术的发展机遇与挑战[J].武汉大学学报(信息科学版), 2018, 43(12):1788-1796. GONG Jianya. Chances and challenges for development of surveying and remote sensing in the age of artificial intelligence[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12):1788-1796.
[12] KERN A, BOBBE M, KHEDAR Y, et al. OpenREALM:real-time mapping for unmanned aerial vehicles[C]//Proceedings of 2020 International Conference on Unmanned Aircraft Systems (ICUAS). Athens:IEEE, 2020:902-911.
[13] LIU Fei, ZHANG Jixian, WANG Jian, et al. An UWB/vision fusion scheme for determining pedestrians'indoor location[J]. Sensors (Basel, Switzerland), 2020, 20(4):1139.
[14] 陈军,刘万增,武昊,等.智能化测绘的基本问题与发展方向[J].测绘学报, 2021, 50(8):995-1005. DOI:10.11947/j.AGCS.2021. 20210235. CHEN Jun, LIU Wanzeng, WU Hao, et al. Smart surveying and mapping:fundamental issues and research agenda[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):995-1005. DOI:10.11947/j.AGCS.2021. 20210235.
[15] 李云天,穆荣军,单永志.无人系统视觉SLAM技术发展现状简析[J].控制与决策, 2021, 36(3):513-522. LI Yuntian, MU Rongjun, SHAN Yongzhi. A survey of visual SLAM in unmanned systems[J]. Control and Decision, 2021, 36(3):513-522.
[16] DAVISON A J, REID I D, MOLTON N D, et al. MonoSLAM:real-time single camera SLAM[J]. IEEE Transactionson Pattern Analysis and Machine Intelligence, 2007, 29(6):1052-1067.
[17] KLEIN G, MURRAY D. Parallel tracking and mapping for small AR workspaces[C]//Proceedings of the 6 th IEEE and ACM International Symposium on Mixed and Augmented Reality. Nara:IEEE, 2007.
[18] PIRE T, FISCHER T, CIVERA J, et al. Stereo parallel tracking and mapping for robot localization[C]//Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Hamburg:IEEE, 2015.
[19] MUR-ARTAL R, MONTIEL J M M, TARDOS J D. ORB-SLAM:a versatile and accurate monocular SLAM system[J]. IEEE Transactions on Robotics, 2015, 31(5):1147-1163.
[20] CAMPOS C, ELVIRA R, RODRÍGUEZ J J G, et al. ORB-SLAM3:an accurate open-source library for visual, visual-inertial, and multimap SLAM[J]. IEEE Transactionson Robotics, 2021, 37(6):1874-1890.
[21] LEUTENEGGER S, LYNEN S, BOSSE M, et al. Keyframe-based visual-inertial odometry using nonlinear optimization[J]. The International Journal of Robotics Research, 2015, 34(3):314-334.
[22] BLOESCH M, OMARI S, HUTTER M, et al. Robust visual inertial odometry using a direct EKF-based approach[C]//Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems. Hamburg:IEEE, 2015.
[23] 刘飞.多传感器融合的高精度无缝定位模型与方法研究[D].徐州:中国矿业大学, 2020. LIU Fei. Research on high-precision seamless positioning model and method based on multi-sensor fusion[D]. Xuzhou:China University of Mining and Technology, 2020.
[24] NEWCOMBE R A, LOVEGROVE S J, DAVISON A J. DTAM:dense tracking and mapping in real-time[C]//Proceedings of 2012 International Conference on Computer Vision. Barcelona:IEEE, 2012:2320-2327.
[25] ENGEL J, SCHÖPS T, CREMERS D. LSD-SLAM:large-scale direct monocular SLAM[C]//Proceedings of 2014 European Conference on Computer Vision. Cham:Springer International Publishing, 2014:834-849.
[26] KERL C, STURM J, CREMERS D. Dense visual SLAM for RGB-D cameras[C]//Proceedings of 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. Tokyo:IEEE, 2014:2100-2106.
[27] CUMMINS M, NEWMAN P. FAB-MAP:probabilistic localization and mapping in the space of appearance[J]. The International Journal of Robotics Research, 2008, 27(6):647-665.
[28] MILFORD M J, WYETH G F. SeqSLAM:visual route-based navigation for sunny summer days and stormy winter nights[C]//Proceedings of 2012 IEEE International Conference on Robotics and Automation. Saint Paul:IEEE, 2012:1643-1649.
[29] SALAS-MORENO R F, NEWCOMBE R A, STRASDAT H, et al. SLAM:simultaneous localisation and mapping at the level of objects[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland:IEEE, 2013:1352-1359.
[30] TATENO K, TOMBARI F, LAINA I, et al. CNN-SLAM:real-time dense monocular SLAM with learned depth prediction[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu:IEEE, 2017.
[31] MILFORD M J, WYETHG F, PRASSERD. RatSLAM:a hippocampal model for simultaneous localization and mapping[C]//Proceedings of 2004 IEEE International Conference on Robotics and Automation. New Orleans:IEEE, 2004.
[32] YU Fangwen, SHANG Jianga, HU Youjian, et al. Neuro SLAM:a brain-inspired SLAM system for 3D environments[J]. Biological Cybernetics, 2019, 113(5):515-545.
[33] 邸凯昌,万文辉,赵红颖,等.视觉SLAM技术的进展与应用[J].测绘学报, 2018, 47(6):770-779. DOI:10.11947/j.AGCS.2018. 20170652. DI Kaichang, WAN Wenhui, ZHAO Hongying, et al. Progress and applications of visual SLAM[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6):770-779. DOI:10.11947/j.AGCS.2018. 20170652.
[34] BU Shuhui, ZHAO Yong, WAN Gang, et al. Map2 DFusion:Real-time incremental UAV image mosaicing based on monocular SLAM[C]//Proceedings of 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Daejeon:IEEE, 2016:4564-4571.
[35] WANG Wei, ZHAO Yong, HAN Pengcheng, et al. TerrainFusion:real-time digital surface model reconstruction based on monocular SLAM[C]//Proceedings of 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Macau:IEEE, 2020:7895-7902.
[36] YANG Tao, LI Peiqi, ZHANG Huiming, et al. Monocular vision SLAM-based UAV autonomous landing in emergencies and unknown environments[J]. Electronics, 2018, 7(5):73.
[37] 潘林豪,田福庆,应文健,等.基于直接法的视觉同时定位与地图构建技术综述[J].计算机应用研究, 2019, 36(4):961-966. PAN Linhao, TIAN Fuqing, YING Wenjian, et al. Survey on direct-method visual simultaneous localization and mapping[J]. Application Research of Computers, 2019, 36(4):961-966.
[38] ENGEL J, STVCKLER J, CREMERS D. Large-scale direct SLAM with stereo cameras[C]//Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Hamburg:IEEE, 2015:1935-1942.
[39] RAGOT N, KHEMMAR R, POKALA A, et al. Benchmark of visual SLAM algorithms:ORB-SLAM2 vs RTAB-map[C]//Proceedings of the 8 th International Conference on Emerging Security Technologies (EST). Colchester:IEEE, 2019:1-6.
[40] FORSTER C, PIZZOLI M, SCARAMUZZA D. SVO:fast semi-direct monocular visual odometry[C]//Proceedings of 2014 IEEE International Conference on Robotics and Automation (ICRA). Hong Kong:IEEE, 2014:15-22.
[41] WANG Rui, SCHWÖRER M, CREMERS D. Stereo DSO:large-scale direct sparse visual odometry with stereo cameras[C]//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). Venice:IEEE, 2017:3923-3931.
[42] MESSINA L, MAZZARO S, FIORILLA A E, et al. Industrial implementation and performance evaluation of LSD-SLAM and map filtering algorithms for obstacles avoidance in a cooperative fleet of unmanned aerial vehicles[C]//Proceedings of the 3 rd International Conference on Intelligent Robotic and Control Engineering (IRCE). Oxford:IEEE, 2020:117-122.
[43] 王敏,付建宽,宗岩,等.高级别自动驾驶汽车计算平台综述[J].时代汽车, 2021(22):30-32. WANG Min, FU Jiankuan, ZONG Yan, et al. Overview of computing platforms for high-level self-driving vehicles[J]. Auto Time, 2021(22):30-32.
[44] LIU Fei, ZHANG Jixian, WANG Jian, et al. The indoor localization of a mobile platform based on monocular vision and coding images[J]. ISPRS International Journal of Geo-Information, 2020, 9(2):122.
[45] SIVIC J, ZISSERMAN A. Video Google:a text retrieval approach to object matching in videos[C]//Proceedings of the 9 th Computer Vision, IEEE International Conference. Nice:IEEE, 2008:1470-1477.
[46] CUMMINS M, NEWMAN P. Appearance-only SLAM at large scale with FAB-MAP 2. 0[J]. The International Journal of Robotics Research, 2011, 30(9):1100-1123.
[47] LOWRY S, SVNDERHAUF N, NEWMAN P, et al. Visual place recognition:a survey[J]. IEEE Transactions on Robotics, 2016, 32(1):1-19.
[48] GARG S, FISCHER T, MILFORD M. Where is your place, visual place recognition?[EB/OL].[2022-04-20]. https://arxiv.org/abs/2103.06443.
[49] MILFORD M J, WYETH G F. SeqSLAM:visual route-based navigation for sunny summer days and stormy winter nights[C]//Proceedings of 2012 IEEE International Conference on Robotics and Automation. Saint Paul:IEEE, 2012:1643-1649.
[50] SIAM S M, ZHANG Hong. Fast-SeqSLAM:a fast appearance based place recognition algorithm[C]//Proceedings of 2017 IEEE International Conference on Robotics and Automation (ICRA). Singapore:IEEE, 2017:5702-5708.
[51] CHANCÁN M, HERNANDEZ-NUNEZ L, NARENDRA A, et al. A hybrid compact neural architecture for visual place recognition[J]. IEEE Robotics and Automation Letters, 2020, 5(2):993-1000.
[52] DOAN D, LATIF Y, CHIN T J, et al. Scalable place recognition under appearance change for autonomous driving[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul:IEEE, 2020:9318-9327.
[53] KOSTAVELIS I, GASTERATOS A. Semantic mapping for mobile robotics tasks:a survey[J]. Robotics and Autonomous Systems, 2015, 66:86-103.
[54] GALINDO C, SAFFIOTTI A, CORADESCHI S, et al. Multi-hierarchical semantic maps for mobile robotics[C]//Proceedings of 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems. Edmonton:IEEE, 2005:2278-2283.
[55] GANTI P. SIVO:semantically informed visual odometry and mapping[D]. Waterloo:University of Waterloo, 2018.
[56] ZHONG Fangwei, WANG Sheng, ZHANG Ziqi, et al. Detect-SLAM:making object detection and SLAM mutually beneficial[C]//Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). Lake Tahoe:IEEE, 2018:1001-1010.
[57] YANG Shichao, SCHERER S. CubeSLAM:monocular 3 D object detection and SLAM without prior models[EB/OL].[2022-03-15]. https://arxiv.org/abs/1806.00557.
[58] BOWMAN S L, ATANASOV N, DANIILIDIS K, et al. Probabilistic data association for semantic SLAM[C]//Proceedings of 2017 IEEE International Conference on Robotics and Automation (ICRA). Singapore. IEEE:2017:1722-1729.
[59] LIANG H J, SANKET N J, FERMVLLER C, et al. SalientDSO:bringing attention to direct sparse odometry[J]. IEEE Transactions on Automation Science and Engineering, 2019, 16(4):1619-1626.
[60] BRASCH N, BOZIC A, LALLEMAND J, et al. Semantic monocular SLAM for highly dynamic environments[C]//Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Madrid:IEEE, 2019:393-400.
[61] FROST D, PRISACARIU V, MURRAY D. Recovering stable scale in monocular SLAM using object-supplemented bundle adjustment[J]. IEEE Transactions on Robotics, 2018, 34(3):736-747.
[62] STENBORG E, TOFT C, HAMMARSTRAND L. Long-term visual localization using semantically segmented images[C]//Proceedings of 2018 IEEE International Conference on Robotics and Automation (ICRA). Brisbane:IEEE, 2018:6484-6490.
[63] 邓晨,李宏伟,张斌,等.基于深度学习的语义SLAM关键帧图像处理[J].测绘学报, 2021, 50(11):1605-1616. DOI:10.11947/j.AGCS.2021. 20210251. DENG Chen, LI Hongwei, ZHANG Bin, et al. Research on key frame image processing of semantic SLAM based on deep learning[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(11):1605-1616. DOI:10.11947/j.AGCS.2021. 20210251.
[64] 李琳辉,张溪桐,连静,等.结合道路结构化特征的语义SLAM算法[J].哈尔滨工业大学学报, 2021, 53(2):175-183. LI Linhui, ZHANG Xitong, LIAN Jing, et al. Semantic SLAM algorithm combined with road structured features[J]. Journal of Harbin Institute of Technology, 2021, 53(2):175-183.
[65] LI Xuanpeng, WANG Dong, AO Huanxuan, et al. Fast 3 D semantic mapping in road scenes[J]. Applied Sciences, 2019, 9(4):631.
[66] ZHOU Yiyang, TAKEDA Y, TOMIZUKA M, et al. Automatic construction of lane-level HD maps for urban scenes[C]//Proceedings of 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Prague:IEEE, 2021:6649-6656.
[67] GARG S, SVNDERHAUF N, DAYOUB F, et al. Semantics for robotic mapping, perception and interaction:a survey[J]. Foundations and Trends^®in Robotics, 2020, 8(1-2):1-224.
[68] 晁丽君,熊智,杨闯,等.无人飞行器三维类脑SLAM自主导航方法[J].飞控与探测, 2020, 3(5):35-43. CHAO Lijun, XIONG Zhi, YANG Chuang, et al. Autonomous navigation method of three-dimension brain-like SLAM for the UAV[J]. Flight Control&Detection, 2020, 3(5):35-43.
[69] 陈孟元.鼠类脑细胞导航机理的移动机器人仿生SLAM综述[J].智能系统学报, 2018, 13(1):107-117. CHEN Mengyuan. Overview of mobile robot bionic slam based on navigation mechanism of mouse brain cells[J]. CAAI Transactions on Intelligent Systems, 2018, 13(1):107-117.
[70] 张潇.基于RatSLAM的微惯性/视觉仿生导航算法研究[D].长沙:国防科学技术大学, 2015. ZHANG Xiao. Research on micro-inertial/visual bionic navigation algorithm based on RatSLAM[D]. Changcha:National University of Defense Technology, 2015.
[71] MILFORD M J, WYETH G F. Mapping a suburb with a single camera using a biologically inspired SLAM system[J]. IEEE Transactions on Robotics, 2008, 24(5):1038-1053.
[72] MILFORD M, WYETH G. Persistent navigation and mapping using a biologically inspired SLAM system[J]. The International Journal of Robotics Research, 2010, 29(9):1131-1153.
[73] LIU Yanqing, ZHU Dongchen, PENG Jingquan, et al. Real-time robust stereo visual SLAM system based on bionic eyes[J]. IEEE Transactions on Medical Robotics and Bionics, 2020, 2(3):391-398.
[74] 蒋晓军.基于鼠脑海马的情景认知地图构建及导航方法研究[D].北京:北京工业大学, 2018. JIANG Xiaojun. Research on the construction of situational cognitive map and navigation method based on rat brain horse[D]. Beijing:Beijing University of Technology, 2018.
[75] LI Junjun, LI Zhijun, CHEN Fei, et al. Combined sensing, cognition, learning, and control for developing future neuro-robotics systems:a survey[J]. IEEE Transactions on Cognitive and Developmental Systems, 2019, 11(2):148-161.
[76] 焦李成,侯彪,唐旭.人工智能、类脑计算与图像解译前沿[M].西安:西安电子科技大学出版社, 2020. JIAO Licheng, HOU Biao, TANG Xu. Frontier of artifical intelligence, brain-like computation, and image processing[M]. Xi'an:Xidian University Press, 2020.
[77] 毛军,付浩,褚超群,等.惯性/视觉/激光雷达SLAM技术综述[J].导航定位与授时, 2022, 9(4):17-30. MAO Jun, FU Hao, CHU Chaoqun, et al. A review of simultaneous localization and mapping based on inertial-visual-LiDAR fusion[J]. Navigation Positioning and Timing, 2022, 9(4):17-30.
[78] 张继贤,刘飞,王坚.轻小型无人机测绘遥感系统研究进展[J].遥感学报, 2021, 25(3):708-724. ZHANG Jixian, LIU Fei, WANG Jian. Review of the light-weighted and small UAV system for aerial photography and remote sensing[J]. Journal of Remote Sensing, 2021, 25(3):708-724.
[79] 宋红霞,张继贤,韩文立,等.基于移动测量系统的导航电子地图质检方法研究[J].测绘地理信息, 2021, 46(2):83-87. SONG Hongxia, ZHANG Jixian, HAN Wenli, et al. Quality inspection of navigation electronic map based on mobile mapping system[J]. Journal of Geomatics, 2021, 46(2):83-87.
[80] NEWCOMBE R A, IZADI S, HILLIGES O, et al. KinectFusion:real-time dense surface mapping and tracking[C]//Proceedings of the 10 th IEEE International Symposium on Mixed and Augmented Reality. Basel:IEEE, 2012:127-136.
[81] 党宇,张继贤,邓喀中,等.基于深度学习AlexNet的遥感影像地表覆盖分类评价研究[J].地球信息科学学报, 2017, 19(11):1530-1537. DANG Yu, ZHANG Jixian, DENG Kazhong, et al. Study on the evaluation of land cover classification using remote sensing images based on AlexNet[J]. Journal of Geo-Information Science, 2017, 19(11):1530-1537.
[82] 张继贤,顾海燕,杨懿,等.高分辨率遥感影像智能解译研究进展与趋势[J].遥感学报, 2021, 25(11):2198-2210. ZHANG Jixian, GU Haiyan, YANG Yi, et al. Research progress and trend of high-resolution remote sensing imagery intelligent interpretation[J]. Journal of Remote Sensing, 2021, 25(11):2198-2210.
[83] 杨启帆,李志茹,马宏忠,等.无人机蜂巢综合系统设计及应用[J].机械研究与应用, 2020, 33(6):167-169. YANG Qifan, LI Zhiru, MA Hongzhong, et al. The UAV honeycomb empowers power inspection[J]. Mechanical Research&Application, 2020, 33(6):167-169