Object-level change detection of multi-sensor optical remote sensing images combined with UNet++ and multi-level difference module

doi:10.11947/j.AGCS.2023.20220202

Abstract

Abstract: With the rapid development of sensor technology, change detection based on multi-sensor optical remote sensing images has become a research hotspot in the field of remote sensing. Due to the differences of sensor imaging, different patterns of manifestation are shown in multi-sensor optical remote sensing images for one scene, leading to a more obvious problem of "pseudo change". Therefore, an object-level change detection method for multi-sensor optical remote sensing images combining UNet++ and multi-stage difference module is proposed in this paper. Firstly, multi-scale feature extraction difference (MFED) module is proposed by this method to enhance the ability of the model to identify "pseudo change". On this basis, multi-scale feature outputs by UNet++ network are used for multi-angle meticulous depiction. Adaptive evidence credibility indicator (AECI) is proposed as well. At last, image segmentation and Dempster-Shafer (DS) theory are combined to design weighted Dempster-Shafer evidence fusion (WDSEF), so as to achieve mapping from pixel-level output of deep network to object-level results. Experiment was conducted to four sets of high-resolution multi-sensor optical image datasets from different regions, and contrastive analysis was conducted to multiple methods of advanced deep learning. The results revealed that, the overall accuracy (OA) and F₁ score of the proposed method reached more than 91.92% and 63.31%, respectively, under different conditions of spatial resolution and temporal phase difference, which were significantly better than the comparison methods in both visual analysis and quantitative evaluation.

Key words: multi-sensor optical remote sensing image, change detection, UNet++, multi-scale feature extraction difference, adaptive evidence reliability index, weighted DS evidence fusion

CLC Number:

P237

WANG Chao, WANG Shuai, CHEN Xiao, LI Junyong, XIE Tao. Object-level change detection of multi-sensor optical remote sensing images combined with UNet++ and multi-level difference module[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(2): 283-296.

References

[1] 黄亮. 多时相遥感影像变化检测技术研究[J]. 测绘学报, 2020, 49(6):801.DOI:10.11947/j.AGCS.2020.20190236. HUANG Liang. Research on change detection technology in multi-temporal remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(6):801.DOI:10.11947/j.AGCS.2020.20190236.
[2] LIU Wensong, JI Xinyuan, LIU Jie, et al. A novel unsupervised change detection method with structure consistency and GFLICM based on UAV images[J]. Journal of Geodesy and Geoinformation Science, 2022, 5(1):91-102.
[3] 许淑淑. 基于对象的多源数据变化检测的方法[J]. 测绘通报, 2020(S1):122-126. XU Shushu. Research on object based multisource data change detection method[J]. Bulletin of Surveying and Mapping, 2020(S1):122-126.
[4] CHEN Dong, WANG Yafei, SHEN Zhenyu, et al. Long time-series mapping and change detection of coastal zone land use based on google earth engine and multi-source data fusion[J]. Remote Sensing,2021, 14(1):1.
[5] CARDILLE J A, PEREZ E, CROWLEY M A, et al. Multi-sensor change detection for within-year capture and labelling of forest disturbance[J]. Remote Sensing of Environment, 2022, 268:112741.
[6] TOUATI R, MIGNOTTE M, DAHMANE M. A reliable mixed-norm-based multiresolution change detector in heterogeneous remote sensing images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(9):3588-3601.
[7] WAN Ling, ZHANG Tao, YOU Hongjian. Object-based method for optical and SAR images change detection[J]. The Journal of Engineering, 2019, 2019(21):7410-7414.
[8] 杨进一, 徐伟铭, 王成军, 等. 基于超像元词包特征和主动学习的高分遥感影像变化检测[J]. 地球信息科学学报, 2019, 21(10):1594-1607. YANG Jinyi, XU Weiming, WANG Chengjun, et al. High-resolution remote sensing imagery change detection based on super-pixel BOW features and active learning[J]. Journal of Geo-Information Science, 2019, 21(10):1594-1607.
[9] 顾炼, 许诗起, 竺乐庆. 基于FlowS-Unet的遥感图像建筑物变化检测[J]. 自动化学报, 2020, 46(6):1291-1300. GU Lian, XU Shiqi, ZHU Leqing. Detection of building changes in remote sensing images via FlowS-unet[J]. Acta Automatica Sinica, 2020, 46(6):1291-1300.
[10] DAUDT R C,LE SAUX B, BOULCH A. Fully convolutional Siamese networks for change detection[C]//Proceedings of 25th IEEE International Conference on Image Processing (ICIP).Athens, Greece:IEEE, 2018:4063-4067.
[11] LIU Jia, GONG Maoguo, QIN Kai, et al. A deep convolutional coupling network for change detection based on heterogeneous optical and radar images[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(3):545-559.
[12] WANG Moyang, TAN Kun, JIA Xiuping, et al. A deep Siamese network with hybrid convolutional feature extraction module for change detection based on multi-sensor remote sensing images[J]. Remote Sensing, 2020, 12(2):205.
[13] PENG Xueli, ZHONG Ruofei, LI Zhen, et al. Optical remote sensing image change detection based on attention mechanism and image difference[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(9):7296-7307.
[14] ZHENG Zhi, WAN Yi, ZHANG Yongjun, et al. CLNet:cross-layer convolutional neural network for change detection in optical remote sensing imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 175:247-267.
[15] 徐炜, 王驿飞, 张艳, 等. 基于双边滤波和小目标抑制的异源遥感变化检测[J]. 电子测量技术, 2021, 44(17):165-172. XU Wei, WANG Yifei, ZHANG Yan, et al. Heterogeneous remote sensing image change detection based on bilateral filtering and small target suppression[J]. Electronic Measurement Technology, 2021, 44(17):165-172.
[16] 周圆, 李祥瑞, 杨晶. 基于混合网络的异源遥感图像变化检测[J]. 北京航空航天大学学报, 2021, 47(3):451-460. ZHOU Yuan, LI Xiangrui, YANG Jing. Heterogeneous remote sensing image change detection based on hybrid network[J]. Journal of Beijing University of Aeronautics and Astronautics, 2021, 47(3):451-460.
[17] SUN Shuting, MU Lin, WANG Lizhe, et al. L-UNet:an LSTM network for remote sensing image change detection[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19:1-5.
[18] LI Lu, WANG Chao, ZHANG Hong, et al. Residual unet for urban building change detection with sentinel-1 SAR data[C]//Proceedings of 2019 IEEE International Geoscience and Remote Sensing Symposium.Yokohama, Japan:IEEE, 2019:1498-1501.
[19] 郝大磊, 肖青, 闻建光, 等. 定量遥感升尺度转换方法研究进展[J]. 遥感学报, 2018, 22(3):408-423. HAO Dalei, XIAO Qing, WEN Jianguang, et al. Advances in upscaling methods of quantitative remote sensing[J]. Journal of Remote Sensing, 2018, 22(3):408-423.
[20] 王威, 杨芳, 张鹏, 等. 基于ENVI/IDL的高分遥感数据自动预处理及植被提取方法:以湖南林业为例[J]. 湖南城市学院学报(自然科学版), 2020, 29(2):45-50. WANG Wei, YANG Fang, ZHANG Peng, et al. An approach for automatic preprocessing of high-resolution remote sensing data and vegetation extraction based on ENVI/IDL:a case study of Hunan forestry[J]. Journal of Hunan City University (Natural Science), 2020, 29(2):45-50.
[21] 呙维, 彭旭, 刘异, 等. 边缘约束下的分形网络分割算法[J]. 武汉大学学报(信息科学版), 2019, 44(11):1693-1699. GUO Wei, PENG Xu, LIU Yi, et al. Edge restricted fractal net evolution approach[J]. Geomatics and Information Science of Wuhan University, 2019, 44(11):1693-1699.
[22] LIN Wenjie, LI Yu, ZHAO Quanhua. High-resolution remote sensing image segmentation using minimum spanning tree tessellation and RHMRF-FCM algorithm[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(1):52-63.
[23] 朱节中, 陈永, 柯福阳, 等. 基于Siam-UNet++的高分辨率遥感影像建筑物变化检测[J]. 计算机应用研究, 2021, 38(11):3460-3465. ZHU Jiezhong, CHEN Yong, KE Fuyang, et al. Building change detection from high resolution remote sensing imagery based on Siam-UNet[J]. Application Research of Computers, 2021, 38(11):3460-3465.
[24] PENG Daifeng, ZHANG Yongjun, GUAN Haiyan. End-to-end change detection for high resolution satellite images using improved UNet++[J]. Remote Sensing, 2019, 11(11):1382.
[25] 廖璐明, 张伟. 基于改进VGG16网络的混合批量训练交通标志识别[J]. 电子科技, 2021, 34(8):8-13. LIAO Luming, ZHANG Wei. Batch mixed training traffic sign recognition based on improved VGG16 network[J]. Electronic Science and Technology, 2021, 34(8):8-13.
[26] WOO S, PARK J, LEE J Y, et al. CBAM:convolutional block attention module[C]//Proceedings of 2018 European conference on computer vision (ECCV).Munich, Germany:Springer International Publishing, 2018:3-19.
[27] ZHOU Zongwei,RAHMAN SIDDIQUEE M M, TAJBAKHSH N, et al. UNet++:A nested U-net architecture for medical image segmentation[C]//Proceedings of 2018 Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support.Granada, Spain:Springer International Publishing, 2018:3-11.
[28] 王艳恒,高连如, 陈正超, 等. 结合深度学习和超像元的高分遥感影像变化检测[J]. 中国图象图形学报, 2020, 25(6):1271-1282. WANG Yanheng, GAO Lianru, CHEN Zhengchao, et al. Deep learning and superpixel-based method for high-resolution remote sensing image change detection[J]. Journal of Image and Graphics, 2020, 25(6):1271-1282.