Progresses and prospects of environment perception and navigation for deep space exploration rovers

doi:10.11947/j.AGCS.2021.20210290

Abstract

Abstract: Environment perception and navigation are the core technologies for automated driving of deep space exploration rovers. Due to the special circumstances of the deep space environment, the rover’s environment perception and navigation are particularly challenging comparing to the corresponding technologies in autonomous car driving on Earth. The paper presents a review of the progresses of environment perception, rover localization and path planning, from engineering application and scientific research perspectives. Moreover, the future prospects of intelligent environment perception and long-range rover navigation are discussed.

Key words: deep space exploration, lunar rover, Mars rover, environment perception, visual localization, path planning

CLC Number:

P227

DI Kaichang, WANG Jia, XING Yan, LIU Zhaoqin, WAN Wenhui, PENG Man, WANG Yexin, LIU Bin, YU Tianyi, LI Lichun, LIU Chuankai. Progresses and prospects of environment perception and navigation for deep space exploration rovers[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(11): 1457-1468.

References

[1] 吴伟仁, 刘继忠, 唐玉华, 等. 中国探月工程[J]. 深空探测学报, 2019, 6(5): 405-416. WU Weiren, LIU Jizhong, TANG Yuhua, et al. China lunar exploration program[J]. Journal of Deep Space Exploration, 2019, 6(5): 405-416.
[2] 邸凯昌,刘召芹,万文辉, 等. 月球和火星遥感制图与探测车导航定位[M]. 北京: 科学出版社, 2015. DI Kaichang, LIU Zhaoqin, WAN Wenhui, et al. Lunar and Mars remote sensing mapping and rover localization[M]. Beijing: Science Press, 2015.
[3] 李群智, 宁远明, 申振荣, 等. 行星表面巡视探测器遥操作技术研究[J]. 航天器工程, 2008, 17(3): 29-35. LI Qunzhi, NING Yuanming, SHEN Zhenrong, et al. Study on teleoperation of rover on planetary surface[J]. SpacecraftEngineering, 2008, 17(3): 29-35.
[4] 邢琰, 滕宝毅, 刘祥, 等. 月球表面巡视探测GNC技术[J]. 空间科学学报, 2016, 36(2): 196-201. XING Yan, TENG Baoyi, LIU Xiang, et al. Guidance, navigation and control technology for lunar surface exploration[J]. Chinese Journal of Space Science, 2016, 36(2): 196-201.
[5] MAIMONE M, JOHNSON A, CHENG Yang, et al. Autonomous navigation results from the Mars exploration rover (MER) mission[M]//ANG M H, KHATIB O. Proceedings of Experimental Robotics IX. Berlin Heidelberg:Springer, 2006, 21: 3-13.
[6] 邸凯昌. 勇气号和机遇号火星车定位方法评述[J]. 航天器工程, 2009, 18(5): 1-5. DI Kaichang. A review of spirit and opportunity rover localization methods[J]. Spacecraft Engineering, 2009, 18(5): 1-5.
[7] ROBERT C, CHARLES J, ROBERT B, et al. Mars Science laboratory participating scientists program proposal information package[R]. Pasadena, CA, USA: The MSL Project Science Office, 2010.
[8] 陈建新, 邢琰, 滕宝毅, 等. 嫦娥三号巡视器GNC及地面试验技术[J]. 中国科学, 2014, 44(5): 461-469. CHEN Jianxin, XING Yan, TENG Baoyi, et al. Guidance, navigation and control technologies of Chang’e-3 lunar rover[J]. SCIENTIA SINICA Technologica, 2014, 44(5): 461-469.
[9] 邢琰, 刘祥, 滕宝毅, 等. 月球表面巡视探测自主局部避障规划[J]. 控制理论与应用, 2019, 36(12): 2042-2046. XING Yan, LIU Xiang, TENG Baoyi, et al. Autonomous local obstacle avoidance path planning of Lunar surface exploration rovers[J]. Control Theory & Applications, 2019, 36(12): 2042-2046.
[10] DI K, LI R. Topographic mapping capability analysis of mars exploration rover 2003 mission imagery[C]//Proceedings of the 5th International Symposium on Mobile Mapping Technology.Kunming,China: [s.n.], 2007.
[11] LI Rongxing, ARVIDSONRE, DI Kaichang, et al. Opportunity rover localization and topographic mapping at the landing site of meridianiplanum,mars[J]. Journal of Geophysical Research: Planets, 2007, 112:E02S90.
[12] LI Rongxing, SQUYRES S W, ARVIDSON R E, et al. Initial results of rover localization and topographic mapping for the 2003 Mars exploration rover mission[J]. Photogrammetric Engineering & Remote Sensing, 2005, 71(10): 1129-1142.
[13] LI Rongxing, ARCHINALB A, ARVIDSONR E, et al. Rover localization and topographic mapping at the landing site of Gusev crater,Mars[J]. Journal of Geophysical Research: Planets, 2006, 111:E02S06.
[14] ALEXANDER D A, DEEN R G, ANDRES P M, et al. Processing of Mars exploration rover imagery for science and operations planning[J]. Journal of Geophysical Research: Planets, 2006, 111:E02S02.
[15] DI Kaichang, XU Fengliang, WANG Jue, et al. Photogrammetric processing of rover imageryof the 2003 Mars Exploration Rover mission[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2008, 63(2): 181-201.
[16] OLSON C F, ABI-RACHED H. Wide-baseline stereo visionfor terrain mapping[J]. Machine Vision and Applications, 2010, 21(5): 713-725.
[17] DI Kaichang, PENG Man. Wide baseline mapping for mars rovers[J]. Photogrammetric Engineering & Remote Sensing, 2011, 77(6): 609-618.
[18] 刘召芹,万文辉, 彭嫚, 等. 遥感制图与导航定位技术在嫦娥三号遥操作中的应用[J]. 遥感学报, 2014, 18(5): 971-980. LIU Zhaoqin, WAN Wenhui, PENG Man, et al. Remote sensing mapping and localization techniques for teleoperation of Chang’e-3 rover[J]. Journal of Remote Sensing, 2014, 18(5): 971-980.
[19] 刘斌, 徐斌, 刘召芹,等. 基于降落相机图像的嫦娥三号着陆轨迹恢复[J]. 遥感学报, 2014, 18(5): 981-987. LIU Bin, XU Bin, LIU Zhaoqin, et al. Descending and landing trajectory recovery of Chang’e-3 lander using descent images[J]. Journal of Remote Sensing, 2014, 18(5): 981-987.
[20] LIU Zhaoqin, DI Kaichang, LI Jian, et al. Landing site topographic mapping and rover localization for Chang’e-4 mission[J]. Science China Information Sciences, 2020, 63: 140901.
[21] WANG Yexin, WAN Wenhui, GOU Sheng, et al. Vision-based decision support for rover path planning in the Chang’e-4 mission[J]. Remote Sensing, 2020, 12(4): 624.
[22] DEEN R G, LORRE J J. Seeing in three dimensions: correlation and triangulation of Mars Exploration Rover imagery[C]//Proceedings of 2005 IEEE International Conference on Systems, Man and Cybernetics.Waikoloa, HI, USA: IEEE, 2005.
[23] WAN W, PENG M, XING Y, et al. A performance comparison of feature detectors for planetary rover mapping and localization[C]//Proceedings of 2017 International Symposium on Planetary Remote Sensing and Mapping.Hong Kong, China: ISPRS, 2017:149-154.
[24] 李海超, 李莹, 顾征. 利用平面约束的月球车立体匹配[J]. 光学精密工程, 2019, 27(2): 433-439. LI Haichao, LI Ying, GU Zheng. Stereo matching method for lunar rover using the plane constraint[J]. Optics and Precision Engineering, 2019, 27(2): 433-439.
[25] 彭嫚, 邸凯昌, 刘召芹. 基于自适应马尔科夫随机场的深空探测影像密集匹配[J]. 遥感学报, 2014, 18(1): 77-89. PENG Man, DI Kaichang, LIU Zhaoqin. Adaptive Markov random field model for dense matching ofdeep space stereo images[J]. Journal of Remote Sensing, 2014, 18(1):77-89.
[26] MATTHIES L, HUERTAS A, CHENG Yang, et al. Stereo vision and shadow analysis for landing hazard detection[C]//Proceedings of 2008 IEEE International Conference on Robotics and Automation.Pasadena, CA, USA: IEEE, 2008.
[27] THOMPSON D, CASTANO R. Performance comparison of rock detection algorithms for autonomous planetary geology[C]//Proceedings of 2007 IEEE Aerospace Conference.Big Sky, MT, USA: IEEE, 2007:1-9.
[28] CASTANO R, ESTLIN T, GAINES D, et al. Onboard autonomous rover science[C]//Proceedings of 2007 IEEE Aerospace Conference. Big Sky, MT, USA: IEEE, 2007.
[29] BAJRACHARYA M. Single image based hazard detection fora planetary lander[C]//Proceedings of the 5th Biannual World Automation Congress.Orlando, FL, USA: IEEE, 2002.
[30] 李东宾, 高宏伟, 王辉. 基于“玉兔号”月面数据的图像分割算法研究[J]. 沈阳理工大学学报, 2017, 36(6): 61-67. LI Dongbin, GAO Hongwei, WANG Hui. Research on image segmentation algorithm based on “Jade Rabbit”data[J]. Transactionsof Shenyang Ligong University, 2017, 36(6): 61-67.
[31] DUNLOP H, THOMPSON D, WETTERGREEN D. Multi-scale features for detection and segmentation of rocks in mars images[C]//Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, MN, USA: IEEE, 2007:1-7.
[32] LI Guoqing, GENG Yunhai, XIAO Xueming. Multi-scale rock detection on Mars[J]. Science China Information Sciences, 2018, 61(10): 102301.
[33] XIAO Xueming, CUI Hutao, YAO Meibao, et al. Autonomous rock detection on mars through region contrast[J]. Advances in Space Research, 2017, 60(3): 626-635.
[34] XIAO Xueming, CUI Hutao, YAO Meibao, et al. Auto rock detection via sparse-based background modeling for mars rover[C]//Proceedings of 2018 IEEE Congress on Evolutionary Computation.Rio de Janeiro, Brazil: IEEE, 2018:1-6.
[35] YANG Juntao, KANG Zhizhong. A gradient-region constrained level set method for autonomous rock detection from mars rover image[C]//Proceedings of 2019 ISPRS Geospatial Week 2019. Enschede, Netherlands:ISPRS, 2019:1479-1485.
[36] VIOLA P, JONES M. Rapid object detection using aboosted cascade of simple features[C]//Proceedings of 2001 IEEEComputer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.Kauai, HI, USA: IEEE, 2001.
[37] FURLÁN F, RUBIO E, SOSSA H, et al. Rock detection in a mars-like environment using a CNN[C]//Proceedings of 2019 Mexican Conference on Pattern Recognition. Querétaro, Mexico: Springer, 2019.
[38] HUERTAS A, CHENG Yang, MATTHIES L H. Automatic hazard detection for landers[C]//Proceedings of the 9th International Symposium on Artificial Intelligence, R & A in Space.Los Angeles, California, USA: [s.n.], 2008.
[39] DI Kaichang, YUE Zongyu, LIU Zhaoqin, et al. Automated rock detection and shape analysis from mars rover imagery and 3D point cloud data[J]. Journal of Earth Science, 2013, 24(1):125-135.
[40] GAO Yang, SPITERI C, PHAM M T, et al. A survey on recent object detection techniques useful for monocular vision-based planetary terrain classification[J]. Robotics and Autonomous Systems, 2014, 62(2): 151-167.
[41] BUE B D, STEPINSKI T F. Automated classification of landforms on Mars[J]. Computers & Geosciences, 2006, 32(5): 604-614.
[42] SHANG C J, BARNES D. Fuzzy-rough feature selection aided support vector machines for Mars image classification[J]. Computer Vision and Image Understanding, 2013, 117(3): 202-213.
[43] ONO M, FUCHS T J, STEFFY A, et al. Risk-aware planetary rover operation: autonomous terrain classification and path planning[C]//Proceedings of 2015 IEEE Aerospace Conference.Big Sky, MT, USA: IEEE, 2015.
[44] ONO M, ROTHROCK B, ALMEIDA E, et al. Data-driven surface traversability analysis for Mars 2020 landing site selection[C]//Proceedings of 2016 IEEE Aerospace Conference. Big Sky, MT, USA: IEEE, 2016.
[45] ROTHROCK B, KENNEDY R, CUNNINGHAM C,et al. SPOC: deep learning-based terrain classification for mars rover missions[C]//Proceedings of 2016 AIAA SPACE 2016.Long Beach, CA, USA: AIAA, 2016: 2016-5539.
[46] ONO M, HEVERLY M, ROTHROCK B, et al. Mars 2020 site-specific mission performance analysis: part 2. Surface traversability[C]//Proceedings of 2018 AIAA SPACE and Astronautics Forum and Exposition. Orlando, FL, USA: AIAA, 2018.
[47] ONO M, ROTHROCK B, OTSU K, et al. MAARS: machine learning-based analytics for automated rover systems[C]//Proceedings of 2020 IEEE Aerospace Conference.Big Sky, MT, USA: IEEE, 2020: 1-17.
[48] REINA G, OJEDA L, MILELLA A, et al. Wheel slippage and sinkage detection for planetary rovers[J]. IEEE/ASME Transactions on Mechatronics, 2006, 11(2): 185-195.
[49] SPITERI C, AL-MILLI S, GAO Y, et al. Real-time visual sinkage detection for planetary rovers[J]. Robotics and Autonomous Systems, 2015, 72: 307-317.
[50] GAO Haibo, LV Fengtian, YUAN Baofeng, et al. Sinkage definition and visual detection for planetary rovers wheels on rough terrain based on wheel-soil interaction boundary[J]. Robotics and Autonomous Systems, 2017, 98: 222-240.
[51] MA Hao, YANG Huan, LI Qunzhi,et al. A geometry-based slip prediction model for planetary rovers[J]. Computers & Electrical Engineering, 2020, 86: 106749.
[52] LI Rongxing, WU Bo, DI Kaichang, et al. Characterization of traverse slippage experienced by Spirit rover on Husband Hill at Gusev crater[J]. Journal of Geophysical Research: Planets, 2008, 113: E12S35.
[53] ARVIDSON R E, DEGROSSE P, GROTZINGER J P, et al. Relating geologic units and mobility system kinematics contributing to Curiosity wheel damage at Gale Crater, Mars[J]. Journal of Terramechanics, 2017, 73: 73-93.
[54] LIU Zhaoqin, DI Kaichang, PENG Man, et al. High precision landing site mapping and rover localization for Chang’e-3 mission[J]. Science China Physics, Mechanics & Astronomy, 2015, 58(1): 1-11.
[55] ALI K, VANELLI C, BIESIADECKI J, et al. Attitude and position estimation on the mars exploration rovers[C]//Proceedings of 2005 IEEE International Conference on System, Man and Cybernetics.Waikoloa, HI, USA: IEEE, 2005.
[56] 王保丰, 周建亮, 唐歌实, 等. 嫦娥三号巡视器视觉定位方法[J].中国科学: 信息科学, 2014, 44(4): 452-460. WANG Baofeng, ZHOU Jianliang, TANG Geshi, et al. Research on visual localization method of lunar rover[J]. Scientia Sinica Informationis, 2014, 44(4): 452-460.
[57] WAN W, LIU Z, DI K, et al. A cross-site visual localization method for Yutu rover[C]//Proceedings of 2014 International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.Suzhou, China: [s.n.], 2014:157-161.
[58] MOREL J M, YU Guoshen. ASIFT: A new framework for fully affine invariant image comparison[J]. SIAM Journal on Imaging Sciences, 2009, 2(2): 438-469.
[59] MAIMONE M W, JOHNSON A E, CHENG Yang, et al. Autonomous navigation results from the Mars exploration rover (MER) Mission[C]//Proceedings of the 9th International Symposium on Experimental Robotics.Singapore:Springer, 2004.
[60] CHENG Yang, MAIMONE M, MATTHIES L. Visual odometry on the Mars exploration rovers: a tool to ensure accurate driving and science imaging[J]. IEEE Robotics & Automation Magazine, 2006, 13(2): 54-62.
[61] MAIMONE M, CHENG Yang, MATTHIES L. Two years of visual odometry on the Mars exploration rovers[J]. Journal of Field Robotics, 2007, 24(3): 169-186.
[62] DI Kaichang, WAN Wenhui, LIU Zhaoqin. New visual odometry method for planetary rover localization[J]. Journal of Remote Sensing, 2012, 17(1): 46-61.
[63] MUR-ARTAL R, MONTIEL J M M, TARDOS J D. ORB-SLAM: a versatile and accurate monocular SLAM system[J]. IEEE Transactions on Robotics, 2015, 31(5): 1147-1163.
[64] ENGEL J, SCHÖPS T, CREMERS D. LSD-SLAM: Large-scale direct monocular SLAM[C]//Proceedings of 2014 European Conference on Computer Vision. Amsterdam, Netherlands: Springer, 2014: 834-849.
[65] LIU Chun, ZHOU Fagen, SUN Yiwei, et al. Stereo-image matching using a speeded up robust feature algorithm in an integrated vision navigation system[J]. The Journal of Navigation, 2012, 65(4): 671-692.
[66] 万文辉, 李宇, 胡文敏, 等. 基于联邦滤波进行立体相机IMU里程计运动平台组合导航定位[J]. 武汉大学学报(信息科学版), 2018, 43(1): 101-106. WAN Wenhui, LI Yu, HU Wenmin, et al. Mobile platform localization by integration of stereo cameras, IMU and wheel qdometer based on federated filter[J]. Geomatics and Information Science of Wuhan University, 2018, 43(1): 101-106.
[67] MOURIKIS A I, ROUMELIOTIS S I. A multi-state constraint Kalman filter for vision-aided inertial navigation[C]//Proceedings of 2007 IEEE International Conference on Robotics and Automation. Rome, Italy: IEEE, 2007: 3565-3572.
[68] LEUTENEGGER S, LYNEN S, BOSSE M, et al. Keyframe-based visual-inertial odometry using nonlinear optimization[J]. The International Journal of Robotics Research, 2015, 34(3): 314-334.
[69] QIN Tong, LI Peiliang, SHEN Shaojie. Vins-mono: a robust and versatile monocular visual-inertial state estimator[J]. IEEE Transactions on Robotics, 2018, 34(4): 1004-1020.
[70] DI Kaichang, WAN Wenhui, ZHAO Hongying, et al. Progress and applications of visual SLAM[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(2): 38-49. DOI: 10.11947/j.JGGS.2019.0205.
[71] DI K, LIU Z, YUE Z. Mars rover localization based on feature matching between ground and orbital imagery[J]. Photogrammetric Engineering & Remote Sensing, 2011, 77(8): 781-791.
[72] CARSTEN J,RANKIN A, FERGUSON D,et al. Global path planning on board the mars exploration rovers[C]//Proceedings of 2007 IEEE Aerospace Conference. Big Sky, MT, USA: IEEE, 2007.
[73] TOMPKINS P, STENTZ A, WETTERGREEN D. Mission-level path planning and re-planning for rover exploration[J]. Robotics and Autonomous Systems, 2006,54(2): 174-183.
[74] LEGER P C, TREBI-OLLENNU A, WRIGHT J R,et al. Mars Exploration Rover surface operations: driving spirit at Gusev Crater[C]//Proceedings of 2005 IEEE International Conference on Systems, Man and Cybernetics. Waikoloa, HI, USA: IEEE, 2005.
[75] TONY T, STENTZ T, WHITTAKER W. Automated surface mission planning considering terrain, shadows, resources and time[C]//Proceedings of the 6th International Symposium on Artificial Intelligence, Robotics and Automation in Space.Montreal, Canada: [s.n.], 2001.
[76] SANCHO-PRADEL D L, GAO Yang. A survey on terrain assessment techniques for autonomous operation of planetary robots[J]. Journal of the British Interplanetary Society, 2010,63(5): 206-217.
[77] 于天一, 费江涛, 李立春, 等. 月面巡视器路径规划方法研究[J]. 深空探测学报, 2019, 6(4): 384-390. YU Tianyi, FEI Jiangtao, LI Lichun, et al. Study on path planning method of lunar rover[J]. Journal of Deep Space Exploration, 2019, 6(4): 384-390.
[78] SASAKI T, OTSU K, THAKKERR,et al.Where to Map? Iterative rover-copter path planning for Mars exploration[J]. IEEE Robotics and Automation Letters, 2020,5(2): 2123-2130.
[79] SUTOH M, OTSUKI M, WAKABAYASHIS,et al.The right path: comprehensive path planning for lunar exploration rovers[J]. IEEE Robotics & Automation Magazine, 2015,22(1): 22-33.
[80] LAVALLE S M. Planning algorithms[M]. Cambridge: Cambridge University Press, 2006.
[81] DELLING D, SANDERS P, SCHULTES D, et al.Engineering route planning algorithms[M]//LERNER J, WAGNER D, ZWEIG K A. Algorithmics of Large and Complex Networks.Berlin, Heidelberg: Springer,2009:117-139.
[82] 刘建军, 陈建新. 基于可通过性的月面巡视探测器路径规划算法[J]. 中国空间科学技术, 2009, 29(3): 16-22. LIU Jianjun, CHEN Jianxin. Traversability based path planning algorithm for lunar rovers[J]. Chinese Space Science and Technology, 2009, 29(3): 16-22.
[83] 郭延宁, 冯振,马广富, 等.行星车视觉导航与自主控制进展与展望[J]. 宇航学报, 2018, 39(11): 1185-1196. GUO Yanning, FENG Zhen, MA Guangfu, et al. Advances and trends in visual navigation and autonomous control of a planetary rover[J]. Journal of Astronautics, 2018, 39(11): 1185-1196.
[84] STENTZ A. Optimal and efficient path planning for partially-known environments[C]//Proceedings of 1994 IEEE International Conference on Robotics and Automation. San Diego, CA, USA: IEEE, 1994.
[85] BASSIL Y. Neural network model for path-planning of robotic rover systems[J].International Journal of Science and Technology, 2012, 2(2):94-100.
[86] BAJRACHARYA M, MAIMONE M W, HELMICK D. Autonomy for mars rovers: past, present, and future[J]. Computer, 2008, 41(12): 44-50.
[87] BIESIADECKI J J, BAUMGARTNER E T, BONITZ R G, et al. Mars exploration rover surface operations: driving opportunity at Meridiani Planum[J]. IEEE Robotics & Automation Magazine, 2006, 13(2): 63-71.
[88] REEVES G E, SNYDER J F. An overview of the Mars exploration rovers’ flight software[C]//Proceedings of 2005 IEEE International Conference on Systems, Man and Cybernetics.Waikoloa, HI, USA: IEEE,2005.