A multi-beam outlier automatic filtering algorithm combining uncertainty and density clustering method

doi:10.11947/j.AGCS.2023.20220579

Abstract

Abstract: Based on the reproduction of CUBE filtering algorithm, this paper proposes a multibeam automatic outlier filtering algorithm combining uncertainty and density clustering method in reference of CUBE's assimilation model. In this paper, we use the DBSCAN to cluster the bathymetry values, use Kalman filter to estimate bathymetry values of the node, and select the bathymetry hypothesis with minimum uncertainty as true bathymetry values of the node. The measured data and simulation results show that the CUBE filtering algorithm cannot completely eliminate the continuous outliers, while the algorithm in this paper can clean up the continuous outliers better. Our algorithm is clear, simple and reliable, and can clean up many outliers in the case of poor data quality, which possesses practical engineering application value.

Key words: multi-beam bathymetry, outlier filter, CUBE algorithm, DBSCAN algorithm, Kalman filter

CLC Number:

P229

WANG Junsen, JIN Shaohua, BIAN Gang, CUI Yang, LONG Zhenyu. A multi-beam outlier automatic filtering algorithm combining uncertainty and density clustering method[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(10): 1669-1678.

References

[1] 赵建虎,刘经南.多波束测深及图像数据处理[M].武汉:武汉大学出版社, 2008. ZHAO Jianhu, LIU Jingnan. Multi-beam sounding and image data processing[M]. Wuhan:Wuhan University Press, 2008.
[2] 纪雪,周兴华,唐秋华,等.多波束测深异常数据检测与剔除方法研究综述[J].测绘科学, 2018, 43(1):38-44. JI Xue, ZHOU Xinghua, TANG Qiuhua, et al. A survey of filtering methods in multibeam bathymetry outliers data[J]. Science of Surveying and Mapping, 2018, 43(1):38-44.
[3] LE DEUNF J, DEBESE N, SCHMITT T, et al. A review of data cleaning approaches in a hydrographic framework with a focus on bathymetric multibeam echosounder datasets[J]. Geosciences, 2020, 10(7):254.
[4] 阳凡林,刘经南,赵建虎.多波束测深数据的异常检测和滤波[J].武汉大学学报(信息科学版), 2004, 29(1):80-83. YANG Fanlin, LIU Jingnan, ZHAO Jianhu. Detecting outliers and filtering noises in multi-beam data[J]. Geomatics and Information Science of Wuhan University, 2004, 29(1):80-83.
[5] 黄贤源,隋立芬,翟国君,等.利用Bayes估计进行多波束测深异常数据探测[J].武汉大学学报(信息科学版), 2010, 35(2):168-171, 214. HUANG Xianyuan, SUI Lifen, ZHAI Guojun, et al. Outliers detection of multi-beam data based on Bayes estimation[J]. Geomatics and Information Science of Wuhan University, 2010, 35(2):168-171, 214.
[6] 董江,任立生.基于趋势面的多波束测深数据滤波方法[J].海洋测绘, 2007, 27(6):25-28. DONG Jiang, REN Lisheng. Filter of MBS sounding data based on trend surface[J]. Hydrographic Surveying and Charting, 2007, 27(6):25-28.
[7] 黄贤源,翟国君,隋立芬,等. LS-SVM算法中优化训练样本对测深异常值剔除的影响[J].测绘学报, 2011, 40(1):22-27. HUANG Xianyuan, ZHAI Guojun, SUI Lifen, et al. The influence of optimized train samples on elimination of sounding outliers in the LS-SVM arithmetic[J]. Acta Geodaetica et Cartographica Sinica, 2011, 40(1):22-27.
[8] 张志衡,彭认灿,黄文骞,等.考虑自然邻点影响域的多波束测深数据趋势面滤波改进算法[J].测绘学报, 2018, 47(1):35-47. DOI:10.11947/j.AGCS.2018.20160565. ZHANG Zhiheng, PENG Rencan, HUANG Wenqian, et al. An improved algorithm of tendency surface filtering in multi-beam bathymetric data considering the natural neighboring points influence field[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(1):35-47. DOI:10.11947/j.AGCS.2018.20160565.
[9] 董箭,彭认灿,张立华,等.利用滚动圆变换的多波束测深数据滤波算法[J].武汉大学学报(信息科学版), 2016, 41(1):86-92. DONG Jian, PENG Rencan, ZHANG Lihua, et al. An algorithm of filtering noises in multi-beam data based on rolling circle transform[J]. Geomatics and Information Science of Wuhan University, 2016, 41(1):86-92.
[10] 杨安秀,吴自银,阳凡林,等.面向多波束测深数据的双向布料模拟自动滤波算法[J].武汉大学学报(信息科学版), 2022, 47(4):517-525, 542. YANG Anxiu, WU Ziyin, YANG Fanlin, et al. An automatic filtering algorithm of multi-beam bathymetry based on bidirectional cloth simulation[J]. Geomatics and Information Science of Wuhan University, 2022, 47(4):517-525, 542.
[11] 黄谟涛,翟国君,王瑞,等.海洋测量异常数据的检测[J].测绘学报, 1999, 28(3):269-277. HUANG Motao, ZHAI Guojun, WANG Rui, et al. The detection of abnormal data in marine survey[J]. Acta Geodaetica et Cartographica Sinica, 1999, 28(3):269-277.
[12] 王乐洋,陈汉清.多波束测深数据处理的抗差最小二乘配置迭代解法[J].测绘学报, 2017, 46(5):658-665. DOI:10.11947/j.AGCS.2017. 20160491. WANG Leyang, CHEN Hanqing. Multi-beam bathymetry data processing using iterative algorithm of robust least squares collocation[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(5):658-665. DOI:10.11947/j.AGCS.2017. 20160491.
[13] 赵荻能,吴自银,李家彪,等. CUBE曲面滤波参数联合优选关键技术及应用[J].测绘学报, 2019, 48(2):245-255. DOI:10.11947/j.AGCS.2019.20180082. ZHAO Dineng, WU Ziyin, LI Jiabiao, et al. The key technology and application of parameter optimization combined CUBE and surface filter[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(2):245-255. DOI:10.11947/j.AGCS.2019. 20180082.
[14] CALDER B. Automatic statistical processing of multibeam echosounder data[J]. International Hydrographic Review, 2003, 4(1):53-68.
[15] CALDER B R, MAYER L A. Robust automatic multi-beam bathymetric processing[C]//Proceedings of 2001 U. S. Hydrographic Conference. Norfolk:Hydrographic Society of America, 2001:21.
[16] CALDER B R, MAYER L A. Automatic processing of high-rate, high-density multibeam echosounder data[J]. Geochemistry, Geophysics, Geosystems, 2003, 4(6):1048-1070.
[17] ZHAO Dineng, WU Ziyin, ZHOU Jieqiong, et al. Parameter group optimization by combining CUBE with surface filering and its application[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(2):81-92.
[18] 黄谟涛,翟国君,柴洪洲,等.检测多波束测深异常数据的CUBE算法模型解析[J].海洋测绘, 2011, 31(4):1-4. HUANG Motao, ZHAI Guojun, CHAI Hongzhou, et al. Analysis on the mathematical models of CUBE algorithm for the detection of abnormal data in multibeam echosounding[J]. Hydrographic Surveying and Charting, 2011, 31(4):1-4.
[19] 王德刚,叶银灿. CUBE算法及其在多波束数据处理中的应用[J].海洋学研究, 2008, 26(2):82-88. WANG Degang, YE Yincan. The theory of CUBE algorithm and its application in the processing of multi-beam data[J]. Journal of Marine Sciences, 2008, 26(2):82-88.
[20] 黄辰虎,陆秀平,侯世喜,等.利用CUBE算法剔除多波束测深粗差研究[J].海洋测绘, 2010, 30(3):1-5. HUANG Chenhu, LU Xiuping, HOU Shixi, et al. Study on detecting outlier of multibeam sounding based on CUBE algorithm[J]. Hydrographic Surveying and Charting, 2010, 30(3):1-5.
[21] 王海栋,柴洪洲.基于CUBE算法的多波束测深数据自动处理研究[J].海洋通报, 2011, 30(3):246-251. WANG Haidong, CHAI Hongzhou. Research on multibeam bathymetry data automatic processing based on CUBE algorithm[J]. Marine Science Bulletin, 2011, 30(3):246-251.
[22] 李沫.基于CUBE多波束异常值检测算法与显控软件[D].哈尔滨:哈尔滨工程大学, 2013. LI Mo. Multi-beam outlier detection algorithm and display control software based on CUBE[D]. Harbin:Harbin Engineering University, 2013.
[23] CALDER B R, WELLS D. CUBE user manual[M]. Durham:University of New Hampshire, 2007.
[24] CALDER B R, RICE G. Design and implementation of an extensible variable resolution bathymetric estimator[C]//Proceedings of 2011 U. S. Hydrographic Conference. Tampa:[s. n.], 2011:25-28.
[25] 王俊森,金绍华,边刚,等. CUBE算法水深假设跟踪模型参数配置方法[J].海洋测绘, 2022, 42(5):14-18. WANG Junsen, JIN Shaohua, BIAN Gang, et al. Parameter configuration method of CUBE algorithm assimilation model[J]. Hydrographic Surveying and Charting, 2022, 42(5):14-18.
[26] 郑永新,张红梅,赵建虎.二次CUBE滤波算法及在边坡乱石区测深数据处理中的应用[J].测绘学报, 2018, 47(11):1549-1557. DOI:10.11947/j.AGCS.2018. 20170354. ZHENG Yongxin, ZHANG Hongmei, ZHAO Jianhu. Second CUBE filtering algorithm and its application in sounding data processing of slope riprap area[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(11):1549-1557. DOI:10.11947/j.AGCS.2018. 20170354.
[27] 蒋齐嘉,蒋中明,唐栋,等.基于SSA-DBSCAN的边坡安全监测数据粗差探测方法[J].长江科学院院报, 2022, 39(4):85-90, 98. JIANG Qijia, JIANG Zhongming, TANG Dong, et al. Gross error detection of slope safety monitoring data based on SSA-DBSCAN[J]. Journal of Yangtze River Scientific Research Institute, 2022, 39(4):85-90, 98.
[28] 雷萌,郭鹏,刘博嵩.基于自适应DBSCAN算法的风电机组异常数据识别研究[J].动力工程学报, 2021, 41(10):859-865. LEI Meng, GUO Peng, LIU Bosong. Study on abnormal data recognition of wind turbines based on adaptive DBSCAN algorithm[J]. Journal of Chinese Society of Power Engineering, 2021, 41(10):859-865.
[29] SEDAGHAT L, HERSEY J, MCGUIRE M P. Detecting spatio-temporal outliers in crowdsourced bathymetry data[C]//Proceedings of the 2 nd ACM SIGSPATIAL International Workshop on Crowdsourced and Volunteered Geographic Information. Orlando:ACM Press, 2013:55-62.
[30] 魏源,金绍华,李树军,等.利用聚类算法实现多波束水深数据异常值的自动识别与清理[J].测绘学报, 2022, 51(11):2294-2302. DOI:10.11947/j.AGCS.2022. 20210113. WEI Yuan, JIN Shaohua, LI Shujun, et al. Automatic recognition and cleaning of outliers in multi-beam bathymetric data with clustering algorithm[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(11):2294-2302. DOI:10.11947/j.AGCS.2022. 20210113.
[31] IHO. Standards for hydrographic surveys 6 th edition:IHO Publication No. 44[S]. Monaco:International Hydrographic Bureau, 2020.
[32] 贾俊涛,申家双,王俊森,等.固定网格的CUBE曲面地形表征能力分析[J].海洋测绘, 2022, 42(4):74-77. JIA Juntao, SHEN Jiashuang, WANG Junsen, et al. Analysis of CUBE surface topography characterization capability with fixed grid[J]. Hydrographic Surveying and Charting, 2022, 42(4):74-77.
[33] HARE R, GODIN A, MAYER L. Accuracy estimation of Canadian swath and sweep sounding systems[R]. Fredericton:Canadian Hydrographic Service, 1995