测绘学报 ›› 2024, Vol. 53 ›› Issue (6): 1195-1211.doi: 10.11947/j.AGCS.2024.20230415
彭代锋1,2,3,4(), 翟晨晨1, 周顶蔚1, 张永军5, 管海燕1, 臧玉府1
收稿日期:
2023-09-28
发布日期:
2024-07-22
作者简介:
彭代锋(1988—),男,博士,副教授,研究方向为遥感影像智能解译。 E-mail:daifeng@nuist.edu.cn
基金资助:
Daifeng PENG1,2,3,4(), Chenchen ZHAI1, Dingwei ZHOU1, Yongjun ZHANG5, Haiyan GUAN1, Yufu ZANG1
Received:
2023-09-28
Published:
2024-07-22
About author:
PENG Daifeng (1988—), male, PhD, associate professor, majors in remote sensing image intelligent interpretation. E-mail: daifeng@nuist.edu.cn
Supported by:
摘要:
针对复杂背景、光谱变化等因素导致高分辨率遥感影像中细小地物检测缺失,几何结构检测不完整等问题,本文联合卷积网络和Transformer网络优势,提出一种基于金字塔语义token全局信息增强的变化检测网络(PST-GIENet)。首先,利用无最大池化层的ResNet18网络提取多时相影像深度特征以构建融合特征,并采用联合注意力机制和深监督策略提高融合特征表达能力;然后,通过空间金字塔池化将影像特征表示为多尺度语义token,进而利用Transformer编码器和解码器对融合特征空间进行全局上下文建模;最后,通过逐层上采样解码器生成最终变化图。为验证本文方法有效性,采用LEVIR-CD、CDD和WHU-CD3个公开变化检测数据集进行对比试验与分析,定量结果表明PST-GIENet在3个数据集中均取得最优精度指标,其F1值分别达到91.71%、96.16%和94.08%。目视结果表明PST-GIENet可有效抑制复杂背景、光谱变化等因素干扰,显著增强网络对地物边缘结构和多尺度变化的捕捉能力,取得最佳目视效果。
中图分类号:
彭代锋, 翟晨晨, 周顶蔚, 张永军, 管海燕, 臧玉府. 基于金字塔语义token全局信息增强的高分光学遥感影像变化检测[J]. 测绘学报, 2024, 53(6): 1195-1211.
Daifeng PENG, Chenchen ZHAI, Dingwei ZHOU, Yongjun ZHANG, Haiyan GUAN, Yufu ZANG. High-resolution optical images change detection based on global information enhancement by pyramid semantic token[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(6): 1195-1211.
表2
不同方法变化检测效果定量比较"
方法 | LEVIR-CD | CDD | WHU-CD | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
P | R | F1值 | IoU | OA | P | R | F1值 | IoU | OA | P | R | F1值 | IoU | OA | |
FC-EF[ | 87.57 | 78.14 | 82.58 | 70.34 | 96.65 | 76.28 | 71.01 | 73.55 | 58.17 | 93.98 | 81.23 | 73.30 | 77.06 | 62.68 | 98.31 |
FC-Siam-Conc[ | 90.96 | 82.96 | 86.78 | 76.64 | 97.35 | 86.21 | 82.36 | 84.24 | 72.77 | 96.36 | 47.88 | 82.84 | 60.69 | 43.56 | 95.85 |
FC-Siam-Diff[ | 91.44 | 79.92 | 85.29 | 74.35 | 97.01 | 83.88 | 79.27 | 81.51 | 68.79 | 95.76 | 79.33 | 72.40 | 75.70 | 60.91 | 98.20 |
UNet++_MSOF[ | 89.08 | 85.37 | 87.19 | 77.28 | 97.31 | 89.36 | 87.22 | 88.27 | 79.00 | 97.27 | 88.96 | 82.27 | 85.48 | 74.65 | 98.92 |
SNUNet[ | 89.98 | 87.63 | 88.79 | 79.83 | 97.79 | 95.45 | 95.14 | 95.29 | 91.01 | 98.89 | 88.58 | 88.68 | 88.63 | 79.58 | 99.12 |
STANet[ | 85.00 | 91.40 | 88.10 | 78.70 | 98.70 | 88.00 | 94.30 | 91.10 | 83.60 | 97.80 | 84.92 | 88.57 | 86.71 | 76.53 | 98.95 |
DSIFN[ | 93.30 | 86.21 | 89.61 | 81.18 | 97.80 | 88.09 | 96.22 | 91.97 | 85.14 | 98.03 | 88.39 | 83.48 | 85.86 | 75.22 | 98.94 |
BIT[ | 90.50 | 89.42 | 89.96 | 81.75 | 98.89 | 95.15 | 92.41 | 93.76 | 88.25 | 98.55 | 90.96 | 89.32 | 90.13 | 82.04 | 99.24 |
ICIF-Net[ | 91.80 | 88.48 | 90.11 | 82.00 | 99.11 | 95.40 | 92.72 | 94.04 | 88.75 | 98.62 | 93.01 | 85.47 | 88.90 | 80.11 | 98.99 |
ChangeFormer[ | 92.05 | 88.81 | 90.40 | 82.48 | 99.04 | 95.32 | 95.50 | 95.41 | 91.22 | 98.92 | 91.59 | 87.72 | 89.62 | 81.18 | 99.21 |
FTNet[ | 92.71 | 89.37 | 91.01 | 83.51 | 99.06 | 92.00 | 77.22 | 83.97 | 72.37 | 96.03 | 95.43 | 89.33 | 92.28 | 85.67 | 99.24 |
PST-GIENet | 92.32 | 91.11 | 91.71 | 84.69 | 99.16 | 95.86 | 96.47 | 96.16 | 92.60 | 99.05 | 95.39 | 92.79 | 94.08 | 88.81 | 99.41 |
表3
LEVIR-CD数据集中各模型精度与复杂度对比"
方法 | F1值/(%) | 参数量 | 测试时间/s |
---|---|---|---|
FC-EF[ | 82.58 | 1.35 | 34 |
FC-Siam-Conc[ | 86.78 | 1.55 | 38 |
FC-Siam-Diff[ | 85.29 | 1.35 | 38 |
UNet++_MSOF[ | 87.19 | 9.05 | 65 |
SNUNet[ | 88.79 | 12.03 | 245 |
DSIFN[ | 88.10 | 50.44 | 117 |
STANet[ | 89.61 | 16.93 | 64 |
BIT[ | 89.96 | 3.49 | 63 |
ICIF-Net[ | 90.11 | 23.84 | 88 |
ChangeFormer[ | 90.40 | 41.01 | 140 |
FTNet[ | 91.01 | 164.45 | 375 |
PST-GIENet | 91.71 | 28.60 | 67 |
表5
Transformer编码器和解码器深度对变化检测精度影响"
ED | DD | LEVIR-CD | CDD | WHU-CD | |||
---|---|---|---|---|---|---|---|
F1值 | IoU | F1值 | IoU | F1值 | IoU | ||
0 | 0 | 91.37 | 84.11 | 95.08 | 90.61 | 92.81 | 86.59 |
1 | 1 | 91.67 | 83.94 | 95.55 | 91.47 | 93.86 | 88.43 |
2 | 1 | 91.40 | 84.16 | 95.58 | 91.54 | 92.70 | 86.39 |
4 | 1 | 91.36 | 84.10 | 95.05 | 90.56 | 92.02 | 85.22 |
8 | 1 | 91.29 | 83.97 | 95.36 | 91.28 | 93.90 | 88.51 |
1 | 2 | 91.60 | 84.51 | 95.59 | 91.54 | 93.56 | 87.90 |
1 | 4 | 91.71 | 84.69 | 95.99 | 92.29 | 93.55 | 87.88 |
1 | 8 | 91.70 | 84.67 | 96.16 | 92.60 | 94.08 | 88.81 |
[1] | BRUZZONE L, BOVOLO F. A novel framework for the design of change-detection systems for very-high-resolution remote sensing images[J]. Proceedings of the IEEE, 2013, 101(3):609-630. |
[2] | TEWKESBURY A P, COMBER A J, TATE N J, et al. A critical synthesis of remotely sensed optical image change detection techniques[J]. Remote Sensing of Environment, 2015, 160:1-14. |
[3] | DEMIR B, BOVOLO F, BRUZZONE L. Updating land-cover maps by classification of image time series: a novel change-detection-driven transfer learning approach[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013, 51(1):300-312. |
[4] | JIN Suming, YANG Limin, DANIELSON P, et al. A comprehensive change detection method for updating the national land cover database to circa 2011[J]. Remote Sensing of Environment, 2013, 132:159-175. |
[5] | LE HÉGARAT-MASCLE S, OTTLÉC , GUÉRIN C. Land cover change detection at coarse spatial scales based on iterative estimation and previous state information[J]. Remote Sensing of Environment, 2005, 95(4):464-479. |
[6] | BRUZZONE L, PRIETO D F. Automatic analysis of the difference image for unsupervised change detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2000, 38(3):1171-1182. |
[7] | FALCO N, MARPU P R, BENEDIKTSSON J A. Comparison of ITPCA and IRMAD for automatic change detection using initial change mask[C]//Proceedings of 2012 IEEE International Geoscience and Remote Sensing Symposium. Munich: IEEE, 2012: 6769-6772. |
[8] | WU Chen, DU Bo, CUI Xiaohui, et al. A post-classification change detection method based on iterative slow feature analysis and Bayesian soft fusion[J]. Remote Sensing of Environment, 2017, 199:241-255. |
[9] | CAO Guo, LI Yupeng, LIU Yazhou, et al. Automatic change detection in high-resolution remote-sensing images by means of level set evolution and support vector machine classification[J]. International Journal of Remote Sensing, 2014, 35(16):6255-6270. |
[10] | CELIK T. Unsupervised change detection in satellite images using principal component analysis and k-means clustering[J]. IEEE Geoscience and Remote Sensing Letters, 2009, 6(4):772-776. |
[11] | BENEDEK C, SZIRANYI T. Change detection in optical aerial images by a multilayer conditional mixed Markov model[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(10):3416-3430. |
[12] | LV Pengyuan, ZHONG Yanfei, ZHAO Ji, et al. Unsupervised change detection based on hybrid conditional random field model for high spatial resolution remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(7):4002-4015. |
[13] | ZHANG Yongjun, PENG Daifeng, HUANG Xu. Object-based change detection for VHR images based on multiscale uncertainty analysis[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(1):13-17. |
[14] | GIL-YEPES J L, RUIZ L A, RECIO J A, et al. Description and validation of a new set of object-based temporal geostatistical features for land-use/land-cover change detection[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 121:77-91. |
[15] | DAI Yuchao, ZHANG Jing, HE Mingyi, et al. Salient object detection from multi-spectral remote sensing images with deep residual network[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(2):101-110. |
[16] | CHAI Junyi, ZENG Hao, LI Anming, et al. Deep learning in computer vision: a critical review of emerging techniques and application scenarios[J]. Machine Learning with Applications, 2021, 6:100134. |
[17] | WANG Min, WANG Peidong. CFM-UNet: a joint CNN and transformer network via cross feature modulation for remote sensing images segmentation[J]. Journal of Geodesy and Geoinformation Science, 2023, 6(4):40-47. |
[18] | YANG Yuanxi, REN Xia, WANG Jianrong. Development of integrated and intelligent geodetic and photogrammetry satellites with corresponding key technologies[J]. Journal of Geodesy and Geoinformation Science, 2023, 6(4):3-12. |
[19] | MA Lei, LIU Yu, ZHANG Xueliang, et al. Deep learning in remote sensing applications: a meta-analysis and review[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 152:166-177. |
[20] | 张祖勋, 姜慧伟, 庞世燕, 等. 多时相遥感影像的变化检测研究现状与展望[J]. 测绘学报, 2022, 51(7):1091-1107. DOI:10.11947/j.AGCS.2022.20220070. |
ZHANG Zuxun, JIANG Huiwei, PANG Shiyan, et al. Review and prospect in change detection of multi-temporal remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(7):1091-1107. DOI:10.11947/j.AGCS.2022.20220070. | |
[21] | EL AMIN A M, LIU Qingjie, WANG Yunhong. Zoom out CNNs features for optical remote sensing change detection[C]//Proceedings of the 2nd International Conference on Image, Vision and Computing. Chengdu: IEEE, 2017: 812-817. |
[22] | ZHANG Hui, GONG Maoguo, ZHANG Puzhao, et al. Feature-level change detection using deep representation and feature change analysis for multispectral imagery[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(11):1666-1670. |
[23] | LEI Yu, LIU Xiaodong, SHI Jiao, et al. Multiscale superpixel segmentation with deep features for change detection[J]. IEEE Access, 2019, 7:36600-36616. |
[24] | GONG Maoguo, ZHAO Jiaojiao, LIU Jia, et al. Change detection in synthetic aperture radar images based on deep neural networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2016, 27(1):125-138. |
[25] | RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]//Proceedings of 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015: 234-241. |
[26] | BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495. |
[27] | CAYE DAUDT R, LE SAUX B, BOULCH A. Fully convolutional Siamese networks for change detection[C]//Proceedings of 2018 IEEE International Conference on Image Processing. Athens: IEEE, 2018: 4063-4067. |
[28] | CAYE DAUDT R, LE SAUX B, BOULCH A, et al. Multitask learning for large-scale semantic change detection[J]. Computer Vision and Image Understanding, 2019, 187:102783. |
[29] | ZHOU Zongwei, SIDDIQUEE M M R, TAJBAKHSH N, et al. UNet++: a nested U-net architecture for medical image segmentation[C]//Proceedings of 2018 Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Springer: Cham, 2018: 3-11. |
[30] | PENG Daifeng, ZHANG Yongjun, GUAN Haiyan. End-to-end change detection for high resolution satellite images using improved UNet++[J]. Remote Sensing, 2019, 11(11):1382. |
[31] | 王超, 王帅, 陈晓, 等. 联合UNet++和多级差分模块的多源光学遥感影像对象级变化检测[J]. 测绘学报, 2023, 52(2):283-296.DOI:10.11947/j.AGCS.2023.20220202. |
WANG Chao, WANG Shuai, CHEN Xiao, et al. Object-level change detection of multi-sensor optical remote sensing images combined with UNet++ and multi-level difference module[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(2):283-296. DOI:10.11947/j.AGCS.2023.20220202. | |
[32] | FANG Sheng, LI Kaiyu, SHAO Jinyuan, et al. SNUNet-CD: a densely connected Siamese network for change detection of VHR images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19:8007805. |
[33] | 叶沅鑫, 孙苗苗, 周亮, 等. 面向建筑物变化检测的主体边缘分解与重组神经网络[J]. 测绘学报, 2023, 52(1):71-81.DOI:10.11947/j.AGCS.2023.20210350. |
YE Yuanxin, SUN Miaomiao, ZHOU Liang, et al. Main body, edge decomposition and reorganization network for building change detection[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(1):71-81. DOI:10.11947/j.AGCS.2023.20210350. | |
[34] | NIU Zhaoyang, ZHONG Guoqiang, YU Hui. A review on the attention mechanism of deep learning[J]. Neurocomputing, 2021, 452:48-62. |
[35] | 梁哲恒, 黎宵, 邓鹏, 等. 融合多尺度特征注意力的遥感影像变化检测方法[J]. 测绘学报, 2022, 51(5):668-676.DOI:10.11947/j.AGCS.2022.20200540. |
LIANG Zheheng, LI Xiao, DENG Peng, et al. Remote sensing image change detection fusion method integrating multi-scale feature attention[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(5):668-676. DOI:10.11947/j.AGCS.2022.20200540. | |
[36] | ZHANG Chenxiao, YUE Peng, TAPETE D, et al. A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 166:183-200. |
[37] | SHI Qian, LIU Mengxi, LI Shengchen, et al. A deeply supervised attention metric-based network and an open aerial image dataset for remote sensing change detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60:5604816. |
[38] | CHEN Jie, YUAN Ziyang, PENG Jian, et al. DASNet: dual attentive fully convolutional Siamese networks for change detection in high-resolution satellite images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14:1194-1206. |
[39] | CHEN Hao, SHI Zhenwei. A spatial-temporal attention-based method and a new dataset for remote sensing image change detection[J]. Remote Sensing, 2020, 12(10):1662. |
[40] | CHEN Hao, QI Zipeng, SHI Zhenwei. Remote sensing image change detection with transformers[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:5607514. |
[41] | SHI Nian, CHEN Keming, ZHOU Guangyao. A divided spatial and temporal context network for remote sensing change detection[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15:4897-4908. |
[42] | SONG Ze, WEI Xiaohui, KANG Xudong, et al. Toward efficient remote sensing image change detection via cross-temporal context learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61:5404210. |
[43] | FENG Yuchao, XU Honghui, JIANG Jiawei, et al. ICIF-net: intra-scale cross-interaction and inter-scale feature fusion network for bitemporal remote sensing images change detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:4410213. |
[44] | BANDARA W G C, PATEL V M. A transformer-based Siamese network for change detection[C]//Proceedings of 2022 IEEE International Geoscience and Remote Sensing Symposium. Kuala Lumpur: IEEE, 2022: 207-210. |
[45] | ZHANG Cui, WANG Liejun, CHENG Shuli, et al. SwinSUNet: pure transformer network for remote sensing image change detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:5224713. |
[46] | YAN Tianyu, WAN Zifu, ZHANG Pingping. Fully transformer network for change detection of remote sensing images[C]//Proceedings of the 16th Asian Conference on Computer Vision. Macao: ACM Press, 2022: 1691-1708. |
[47] | WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of 2018 European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 3-19. |
[48] | LEE C Y, XIE S, GALLAGHER P, et al. Deeply-supervised nets[C]//Proceedings of 2015 Conference on Artificial Intelligence and Statistics. [S.l.]: PMLR, 2015: 562-570. |
[49] | LEBEDEV M A, VIZILTER Y V, VYGOLOV O V, et al. Change detection in remote sensing images using conditional adversarial networks[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2018, XLII-2:565-571. |
[50] | JI Shunping, WEI Shiqing, LU Meng. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(1):574-586. |
[1] | 殷吉崇, 武芳, 翟仁健, 邱越, 巩现勇, 行瑞星. 面向建筑物轮廓规则化的双路径边界约束与相对论生成对抗网络[J]. 测绘学报, 2024, 53(7): 1444-1457. |
[2] | 宁晓刚, 张翰超, 张瑞倩. 遥感影像高可信智能不变检测技术框架与方法实践[J]. 测绘学报, 2024, 53(6): 1098-1112. |
[3] | 蒋亚楠, 郑林枫, 许强, 汤明高, 朱星. 机理引导下的阶跃型滑坡位移预测深度学习模型[J]. 测绘学报, 2024, 53(6): 1128-1139. |
[4] | 顾海燕, 杨懿, 李海涛, 孙立坚, 丁少鹏, 刘世琦. 高分辨率遥感影像样本库动态构建与智能解译应用[J]. 测绘学报, 2024, 53(6): 1165-1179. |
[5] | 王继成, 郭安嵋, 慎利, 蓝天, 徐柱, 李志林. 多级对比学习下的弱监督高分遥感影像城市固废堆场提取[J]. 测绘学报, 2024, 53(6): 1212-1223. |
[6] | 丁少鹏, 卢秀山, 刘如飞, 杨懿, 顾海燕, 李海涛. 联合目标特征引导与多重注意力的建筑物变化检测[J]. 测绘学报, 2024, 53(6): 1224-1235. |
[7] | 纪长琦, 郭肇捷, 孙海丽, 钟若飞. 基于移动激光扫描的地铁隧道渗漏水定位及快速检测方法[J]. 测绘学报, 2024, 53(6): 1236-1250. |
[8] | 林云浩, 王艳军, 李少春, 蔡恒藩. 一种耦合DeepLab与Transformer的农作物种植类型遥感精细分类方法[J]. 测绘学报, 2024, 53(2): 353-366. |
[9] | 江宝得, 黄威, 许少芬, 巫勇. 融合分散自适应注意力机制的多尺度遥感影像建筑物实例细化提取[J]. 测绘学报, 2023, 52(9): 1504-1514. |
[10] | 韦春桃, 龚成, 周永绪. 一种联合空间约束与差异特征聚合的变化检测网络[J]. 测绘学报, 2023, 52(9): 1538-1547. |
[11] | 张艺超, 郑向涛, 卢孝强. 基于层级Transformer的高光谱图像分类方法[J]. 测绘学报, 2023, 52(7): 1139-1147. |
[12] | 胡功明, 杨春成, 徐立, 尚海滨, 王泽凡, 秦志龙. 改进U-Net的遥感图像语义分割方法[J]. 测绘学报, 2023, 52(6): 980-989. |
[13] | 顾小虎, 李正军, 缪健豪, 李星华, 沈焕锋. 高分遥感影像双通道并行混合卷积分类方法[J]. 测绘学报, 2023, 52(5): 798-807. |
[14] | 胡明洪, 李佳田, 姚彦吉, 阿晓荟, 陆美, 李文. 结合多路径的高分辨率遥感影像建筑物提取SER-UNet算法[J]. 测绘学报, 2023, 52(5): 808-817. |
[15] | 余东行, 徐青, 赵传, 郭海涛, 卢俊, 林雨准, 刘相云. 注意力引导特征融合与联合学习的遥感影像场景分类[J]. 测绘学报, 2023, 52(4): 624-637. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||