语素关联约束的动态环境视觉定位优化

doi:10.11947/j.AGCS.2021.20210308

测绘学报 ›› 2021, Vol. 50 ›› Issue (11): 1478-1486.doi: 10.11947/j.AGCS.2021.20210308

语素关联约束的动态环境视觉定位优化

邵晓航, 吴杭彬, 刘春, 陈晨, 蔡天池, 程帆瑾

同济大学测绘与地理信息学院, 上海 200092

收稿日期:2021-05-31 修回日期:2021-08-30 发布日期:2021-12-07
通讯作者: 刘春 E-mail:liuchun@tongji.edu.cn
作者简介:邵晓航(1992—),男,博士生,研究方向为视觉机器人定位与环境认知。
基金资助:
“十三五”重点研发计划（2018YFB1305003）；国家自然科学基金（41771481）；中央高校基本科研学科交叉重点项目（22120190195）

Visual odometry optimizing bounded with semantic elements association in dynamic scenes

SHAO Xiaohang, WU Hangbin, LIU Chun, CHEN Chen, CAI Tianchi, CHENG Fanjin

College of surveying and Geo-informatics, Tongji university, Shanghai 200092, China

Received:2021-05-31 Revised:2021-08-30 Published:2021-12-07
Supported by:
The National Science and Technology Major Program (No. 2018YFB1305003);The National Natural Science Foundation of China (No. 41771481);The Fundamental Research of Interdisciplinary Program for Central Universities (No. 22120190195)

摘要/Abstract

摘要： 在自动驾驶场景中，视觉相机能够实现低成本的定位与环境感知，但是场景中的动态目标会影响视觉定位的轨迹。对此，本文提出了语素关联约束的动态环境视觉定位优化方法。首先，利用目标检测和语义分割提取环境中的语义实体；然后，通过语素关联模型识别出动态语素；最后，建立动态语素的特征掩膜，用于特征匹配过程中的动态目标特征点过滤，从而提高视觉定位效果。本文基于视觉机器人平台在校园道路开展了试验，发现了动态目标通过关键点影响视觉定位结果的规律——在转弯时或者目标在视野中横向移动时影响较大。试验结果表明，本文方法的动态语义要素识别的平均精度F₁约为87%，并且在语素关联优化前后，局部区域最大轨迹距离差为2.463 m，与真值对比RMSE降低了38%。

关键词: 视觉定位, 同时定位与地图构建, 动态场景, 语义关联, 语义分割

Abstract: Cameras can help to make low-cost positioning and environment perceiving in autopilot but dynamic objects give negative effects to visual odometry. This paper gives a model to optimize it based on semantic elements association. It uses such techniques as objects detection and semantic segmentation to identify semantic objects and distinguish dynamic semantic elements(DSE) from static ones. Then it filters bad keypoints by a dynamic feature mask in visual positioning. In practice, this proposed method detects DSE even when there are objects with duality of moving and static. In an experiment on campus roads, negative influences were found especially when a robot turning around or a moving object crossing its camera view. It showed that average accuracy in detecting DSE was 87% and the largest difference between trajectories before and after semantic association optimizing in sub-sequences was 2.463 m. Compared to ground truth, the RMSE of proposed method dropped by 38%.

Key words: visual odometry, SLAM, dynamic scenes, semantic association, semantic segmentation

中图分类号:

P237

邵晓航, 吴杭彬, 刘春, 陈晨, 蔡天池, 程帆瑾. 语素关联约束的动态环境视觉定位优化[J]. 测绘学报, 2021, 50(11): 1478-1486.

SHAO Xiaohang, WU Hangbin, LIU Chun, CHEN Chen, CAI Tianchi, CHENG Fanjin. Visual odometry optimizing bounded with semantic elements association in dynamic scenes[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(11): 1478-1486.

参考文献

[1] PADEN B, ČÁP M, YONG S Z, et al. A survey of motion planning and control techniques for self-driving urban vehicles[J]. IEEE Transactions on Intelligent Vehicles, 2016, 1(1): 33-55.
[2] 张一, 姜挺, 江刚武, 等. 特征法视觉SLAM逆深度滤波的三维重建[J]. 测绘学报, 2019, 48(6): 708-717. DOI: 10.11947/j.AGCS.2019.20180421. ZHANG Yi, JIANG Ting, JIANG Gangwu, et al. 3D reconstruction with inverse depth filter of feature-based visual SLAM[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6): 708-717. DOI: 10.11947/j.AGCS.2019.20180421.
[3] 王晨捷, 罗斌, 李成源, 等. 无人机视觉SLAM协同建图与导航[J]. 测绘学报, 2020, 49(6): 767-776. DOI: 10.11947/j.AGCS.2020.20190145. WANG Chenjie, LUO Bin, LI Chengyuan, et al. The collaborative mapping and navigation based on visual SLAM in UAV platform[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(6): 767-776. DOI: 10.11947/j.AGCS.2020.20190145.
[4] KLEIN G, MURRAY D. Parallel tracking and mapping for small AR workspaces[C]//Proceedings of the 6th IEEE and ACM International Symposium on Mixed and Augmented Reality. Nara, Japan: IEEE, 2007: 225-234.
[5] MUR-ARTAL R, MONTIEL J M M, TARDÓS J D. ORB-SLAM: a versatile and accurate monocular SLAM system[J]. IEEE Transactions on Robotics, 2015, 31(5): 1147-1163.
[6] MUR-ARTALR, TARDÓS J D. ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017, 33(5): 1255-1262.
[7] CAMPOS C,ELVIRA R, RODRÍGUEZ J J G, et al. ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap SLAM[J]. IEEE Transactions on Robotics, 2021, 37(1): 1-17.
[8] NEWCOMBE R A, LOVEGROVE S J, DAVISON A J. DTAM: Dense tracking and mapping in real-time[C]//Proceedings of 2011 International Conference on Computer Vision. Barcelona, Spain: IEEE, 2011: 2320-2327.
[9] FORSTER C, PIZZOLI M, SCARAMUZZA D. SVO: Fast semi-direct monocular visual odometry[C]//Proceedings of 2014 IEEE International Conference on Robotics and Automation (ICRA). Hong Kong, China: IEEE, 2014: 15-22.
[10] BAHRAINI M S, BOZORG M, RAD A B. SLAM in dynamic environments via ML-RANSAC[J]. Mechatronics, 2018, 49: 105-118.
[11] STRECKE M, STUCKLER J. EM-fusion: Dynamic object-level SLAM with probabilistic data association[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South): IEEE, 2019: 5864-5873.
[12] 弋英民, 刘丁. 动态环境下基于路径规划的机器人同步定位与地图构建[J]. 机器人, 2010, 32(1): 83-90. YI Yingmin, LIU Ding. Robot simultaneous localization and mapping based on path planning in dynamic environments[J]. Robot, 2010, 32(1): 83-90.
[13] ÖZKAN E, WAHLSTRÖM N, GODSILL S J. Rao-Blackwellised particle filter for star-convex extended target tracking models[C]//Proceedings of the 19th International Conference on Information Fusion (FUSION). Heidelberg, Germany: IEEE, 2016: 1193-1199.
[14] 龚健雅, 季顺平. 摄影测量与深度学习[J]. 测绘学报, 2018, 47(6): 693-704. DOI: 10.11947/j.AGCS.2018.20170640. GONG Jianya, JI Shunping. Photogrammetry and deep learning[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6): 693-704. DOI: 10.11947/j.AGCS.2018.20170640.
[15] 邸凯昌, 万文辉, 赵红颖, 等. 视觉SLAM技术的进展与应用[J]. 测绘学报, 2018, 47(6): 770-779. DOI: 10.11947/j.AGCS.2018.20170652. DI Kaichang, WAN Wenhui, ZHAO Hongying, et al. Progress and applications of visual SLAM[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6): 770-779. DOI: 10.11947/j.AGCS.2018.20170652.
[16] YU C, LIU Z X, LIU X J, et al. DS-SLAM: a semantic visual SLAM towards dynamic environments[C]//Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Madrid, Spain: IEEE, 2018: 1168-1174.
[17] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
[18] BESCOS B, FÁCIL J M, CIVERA J, et al. DynaSLAM: tracking, mapping, and inpainting in dynamic scenes[J]. IEEE Robotics and Automation Letters, 2018, 3(4): 4076-4083.
[19] HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017: 2980-2988.
[20] RUNZ M, BUFFIER M, AGAPITO L. MaskFusion: real-time recognition, tracking and reconstruction of multiple moving objects[C]//Proceedings of 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). Munich, Germany: IEEE, 2018: 10-20.
[21] 李琳辉, 张溪桐, 连静, 等. 结合道路结构化特征的语义SLAM算法[J]. 哈尔滨工业大学学报, 2021, 53(2): 175-183. LI Linhui, ZHANG Xitong, LIAN Jing, et al. Semantic SLAM algorithm combined with road structured features[J]. Journal of Harbin Institute of Technology, 2021, 53(2): 175-183.
[22] ZHONG F W, WANG S, ZHANG Z Q, et al. Detect-SLAM: Making object detection and SLAM mutually beneficial[C]//Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). Lake Tahoe, NV, USA: IEEE, 2018: 1001-1010.
[23] KONOLIGE K, AGRAWAL M. FrameSLAM: From bundle adjustment to real-time visual mapping[J]. IEEE Transactions on Robotics, 2008, 24(5): 1066-1077.
[24] WESTOBY M J, BRASINGTON J, GLASSER N F, et al. “Structure-from-Motion” photogrammetry: a low-cost, effective tool for geoscience applications[J]. Geomorphology, 2012, 179: 300-314.
[25] 黄瑞, 张轶. 高适应性激光雷达SLAM[J]. 电子科技大学学报, 2021, 50(1): 52-58. HUANG Rui, ZHANG Yi. High adaptive LiDAR simultaneous localization and mapping[J]. Journal of University of Electronic Science and Technology of China, 2021, 50(1): 52-58.
[26] HULETSKI A, KARTASHOV D, KRINKIN K. Evaluation of the modern visual SLAM methods[C]//Proceedings of 2015 Artificial Intelligence and Natural Language and Information Extraction, Social Media and Web Search FRUCT Conference (AINL-ISMW FRUCT). St. Petersburg, Russia: IEEE, 2015: 19-25.

语素关联约束的动态环境视觉定位优化

Visual odometry optimizing bounded with semantic elements association in dynamic scenes

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	赵玏洋, 闫利. 移动机器人SLAM位姿估计的改进四元数无迹卡尔曼滤波[J]. 测绘学报, 2022, 51(2): 212-223.
[2]	蒋腾平, 王永君, 张林淇, 梁冲, 孙剑. 融合CNN和MRF的激光点云层次化语义分割方法[J]. 测绘学报, 2021, 50(2): 215-225.
[3]	张星, 林静, 李清泉, 刘涛, 方志祥. 结合感知哈希与空间约束的室内连续视觉定位方法[J]. 测绘学报, 2021, 50(12): 1639-1649.
[4]	邸凯昌, 王镓, 邢琰, 刘召芹, 万文辉, 彭嫚, 王晔昕, 刘斌, 于天一, 李立春, 刘传凯. 深空探测车环境感知与导航定位技术进展与展望[J]. 测绘学报, 2021, 50(11): 1457-1468.
[5]	李帅鑫, 李九人, 田滨, 陈龙, 王力, 李广云. 面向点云退化的隧道环境的无人车激光SLAM方法[J]. 测绘学报, 2021, 50(11): 1487-1499.
[6]	许智宾, 李宏伟, 张斌, 肖志远, 邓晨. 双目视觉与惯导融合的移动机器人定位方法[J]. 测绘学报, 2021, 50(11): 1512-1521.
[7]	周哲, 胡钊政, 李娜, 肖汉彪, 伍锦祥. 面向智能车的地下停车场环视特征地图构建与定位[J]. 测绘学报, 2021, 50(11): 1574-1584.
[8]	孙喜亮, 关宏灿, 苏艳军, 徐光彩, 郭庆华. 面向高精度城市测绘的激光紧耦合SLAM方法[J]. 测绘学报, 2021, 50(11): 1585-1593.
[9]	李道纪, 郭海涛, 卢俊, 赵传, 林雨准, 余东行. 遥感影像地物分类多注意力融和U型网络法[J]. 测绘学报, 2020, 49(8): 1051-1064.
[10]	姚连璧, 秦长才, 张邵华, 陈启超, 阮东旭, 聂顺根. 车载激光点云的道路标线提取及语义关联[J]. 测绘学报, 2020, 49(4): 480-488.
[11]	耿艳磊, 陶超, 沈靖, 邹峥嵘. 高分辨率遥感影像语义分割的半监督全卷积网络法[J]. 测绘学报, 2020, 49(4): 499-508.
[12]	刘嗣超, 武鹏达, 赵占杰, 李成名. 交通监控视频图像语义分割及其拼接方法[J]. 测绘学报, 2020, 49(4): 522-532.
[13]	王忠武, 王志盼, 尤淑撑, 雷帆, 曹里, 杨凯钧. Landsat影像冰川提取的上下文感知语义分割网络法[J]. 测绘学报, 2020, 49(12): 1575-1582.
[14]	徐旺, 游雄, 张威巍, 邓晨. 2D地图的建筑物场景结构提取方法及其在城市增强现实中的应用[J]. 测绘学报, 2020, 49(12): 1619-1629.
[15]	李雪, 张力, 王庆栋, 艾海滨. 多时相遥感影像语义分割色彩一致性对抗网络[J]. 测绘学报, 2020, 49(11): 1473-1484.