A high-resolution remote sensing images change detection method via the integration of dense connections and self-attention mechanisms

doi:10.11947/j.AGCS.2024.20230454

Abstract

Abstract:

Remote sensing image change detection is an important task in remote sensing image analysis, which is widely used in urban dynamic monitoring, geographic information update, natural disaster monitoring, illegal building investigation, military target strike effect analysis, and land and resources survey. As a pixel-level prediction task, the current methods of change detection have two prominent problems: one is the computational efficiency of self-attention between arbitrary pixel pairs is low, and the long context information in remote sensing images is insufficiently utilized; the other is that the current methods focus on the extraction of deep change image features while shallow information containing high-resolution and fine-grained features are ignored. To address the first problem, Transformer is used to perform context modeling on the extracted bitemporal image features to improve the quality of the deepest change features. To take into account the efficiency of the Transformer, the proposed method converts the images into sparse tokens, thereby significantly reducing the number of tokens of the Transformer. For the second problem, the proposed method uses dense skip connections to retain high resolution in shallow change features. Three publicly available datasets were used for experiments. Extensive experiments show that compared with the state-of-the-art change detection methods, the IoU metric of the proposed method reached 85.44%, 84.15% and 94.61%, respectively, which is better than other comparison methods.

Key words: remote sensing images, change detection, Transformer, dense connections, multi-scale feature fusion

CLC Number:

P237

Shiyan PANG, Jingjing HAO, Zhiqi ZUO, Jingjing LAN, Xiangyun HU. A high-resolution remote sensing images change detection method via the integration of dense connections and self-attention mechanisms[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(12): 2244-2253.

Figures/Tables 12

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Tab. 1

Fig. 8

Tab. 2

Tab. 3

Fig. 9

References 29

[1]	FANG S, LI K, SHAO J, et al. SNUNet-CD: a densely connected Siamese network for change detection of VHR images[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 19: 1-5.
[2]	LIU W, JI X, LIU J, et al. A novel unsupervised change detection method with structure consistency and GFLICM based on UAV images[J]. Journal of Geodesy and Geoinformation Science, 2022, 5(1): 91-102.
[3]	张祖勋, 姜慧伟, 庞世燕, 等. 多时相遥感影像的变化检测研究现状与展望[J]. 测绘学报, 2022, 51(7): 1091-1107. DOI:. doi: 10.11947/j.AGCS.2022.20220070
	ZHANG Zuxun, JIANG Huiwei, PANG Shiyan, et al. Review and prospect in change detection of multi-temporal remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(7): 1091-1107. DOI:. doi: 10.11947/j.AGCS.2022.20220070
[4]	ZAGORUYKO S, KOMODAKIS N. Learning to compare image patches via convolutional neural networks[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 4353-4361.
[5]	ZHAN Yang, FU Kun, YAN Menglong, et al. Change detection based on deep siamese convolutional network for optical aerial images[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(10): 1845-1849.
[6]	DAUDT R C, LE SAUX B, BOULCH A. Fully convolutional siamese networks for change detection[C]//Proceedings of 2018 IEEE International Conference on Image Processing. Athens: IEEE, 2018: 4063-4067.
[7]	ZHANG Wuxia, LU Xiaoqiang. The spectral-spatial joint learning for change detection in multispectral imagery[J]. Remote Sensing, 2019, 11(3): 240.
[8]	CHEN Hao, LI Wenyuan, SHI Zhenwei. Adversarial instance augmentation for building change detection in remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60: 1-16.
[9]	PENG Daifeng, ZHANG Yongjun, GUAN Haiyan. End-to-end change detection for high resolution satellite images using improved UNet++[J]. Remote Sensing, 2019, 11(11): 1382.
[10]	HUANG Gao, LIU Zhuang, VAN DER M L, et al. Densely connected convolutional networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 4700-4708.
[11]	ZHOU Zongwei, SIDDIQUEE M M R, TAJBAKHSH N, et al. UNet++: redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 39(6): 1856-1867.
[12]	王超, 王帅, 陈晓, 等. 联合UNet++和多级差分模块的多源光学遥感影像对象级变化检测[J]. 测绘学报, 2023, 52(2): 283-296. DOI:. doi: 10.11947/j.AGCS.2023.20220202
	WANG Chao, WANG Shuai, CHEN Xiao, et al. Object-level change detection of multi-sensor optical remote sensing images combined with UNet++ and multi-level difference module[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(2): 283-296. DOI:. doi: 10.11947/j.AGCS.2023.20220202
[13]	梁哲恒, 黎宵, 邓鹏, 等. 融合多尺度特征注意力的遥感影像变化检测方法[J]. 测绘学报, 2022, 51(5): 668-676. DOI:. doi: 10.11947/j.AGCS.2022.20200540
	LIANG Zheheng, LI Xiao, DENG Peng, et al. Remote sensing image change detection fusion method integrating multi-scale feature attention[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(5): 668-676. DOI:. doi: 10.11947/j.AGCS.2022.20200540
[14]	ZHANG Chenxiao, YUE Peng, TAPETE D, et al. A deeply supervised image fusion network for change detection in high resolution bitemporal remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 166: 183-200.
[15]	PENG Xueli, ZHONG Ruofei, LI Zhen, et al. Optical remote sensing image change detection based on attention mechanism and image difference[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59(9): 7296-7307.
[16]	CHEN Hao, SHI Zhenwei. A spatial-temporal attention-based method and a new dataset for remote sensing image change detection[J]. Remote Sensing, 2020, 12(10): 1662-1684.
[17]	JIANG Huiwei, HU Xiangyun, LI Kun, et al. PGA-SiamNet: pyramid feature-based attention-guided Siamese network for remote sensing orthoimagery building change detection[J]. Remote Sensing, 2020, 12(3): 484.
[18]	SHI Qian, LIU Mengxi, LI Shengchen, et al. A deeply supervised attention metric-based network and an open aerial image dataset for remote sensing change detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60: 1-16.
[19]	LIU Yi, PANG Chao, ZHAN Zongqian, et al. Building change detection for remote sensing images using a dual-task constrained deep siamese convolutional network model[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 18(5): 811-815.
[20]	ZHANG Lin, HU Xiangyun, ZHANG Mi, et al. Object-level change detection with a dual correlation attention-guided detector[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 177, 147-160.
[21]	CHEN Pan, ZHANG Bing, HONG Danfeng, et al. FCCDN: feature constraint network for VHR image change detection[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 187: 101-119.
[22]	FANG Sheng, LI Kaiyu, LI Zhe. Changer: feature interaction is what you need for change detection[EB/OL]. [2023-09-01]. https://arxiv.org/abs/2209.08290v1.
[23]	CHEN Hao, QI Zipeng, SHI Zhenwei. Remote sensing image change detection with transformers[J]. IEEE Transactions on Geos-cience and Remote Sensing, 2021, 60: 1-14.
[24]	BANDARA W G C, PATEL V M. A Transformer-based Siamese network for change detection[C]//Proceedings of 2022 IEEE International Geoscience and Remote Sensing Symposium. Kuala Lumper: IEEE, 2022: 207-210.
[25]	ZHENG Zhuo, ZHONG Yanfei, TIAN Shiqi, et al. ChangeMask: deep multi-task Encoder-Transformer-Decoder architecture for semantic change detection[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 183: 228-239.
[26]	CHEN Zhanlong, ZHOU Yuan, WANG Bin, et al. EGDE-Net: a building change detection method for high-resolution remote sensing imagery based on edge guidance and differential enhancement[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 191, 203-222.
[27]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[28]	DENG Jia, DONG Wei, SOCHER R, et al. Imagenet: a large-scale hierarchical image database[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE, 2009: 248-255.
[29]	SELVARAJU R R, COGSWELL M, DAS A, et al. Gradcam: visual explanations from deep networks via gradient-based localization[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 618-626.

模型	WHU-CD						LEVIR-CD						CDD
模型	IoU	OA	Precision	Recall	F₁值	Kappa	IoU	OA	Precision	Recall	F₁值	Kappa	IoU	OA	Precision	Recall	F₁值	Kappa
DTCDSCN	82.19	99.31	92.70	87.87	90.22	89.86	79.90	98.86	89.04	88.61	88.82	88.23	86.15	98.22	94.08	91.08	92.56	91.55
SNUNet	79.09	99.17	90.68	86.08	88.32	87.89	82.16	98.97	94.83	86.01	90.21	85.76	88.13	98.47	93.99	93.39	93.69	92.82
BIT	81.29	99.26	90.66	88.71	89.68	89.29	82.63	98.99	93.88	87.33	90.49	89.95	89.19	98.62	95.12	93.47	94.29	93.50
ChangeFormer	76.28	99.07	92.11	81.62	86.55	86.07	78.21	98.78	89.74	85.89	87.77	87.13	84.53	98.00	94.99	88.47	91.62	90.48
ChangerEx	72.09	98.77	80.43	87.42	83.78	83.14	81.62	98.99	91.76	88.08	89.88	89.35	89.52	98.67	96.82	92.24	94.47	93.71
TNUNet-CD	85.44	99.44	93.61	90.74	92.15	91.86	84.15	99.13	92.10	90.70	91.40	90.94	94.61	99.33	97.97	96.50	97.23	96.85

模型	Params/MB	FLOPs/G
DTCDSCN	31.26	52.89
SNUNet	12.03	219.30
BIT	24.74	232.14
ChangeFormer	41.02	810.50
ChangerEx	11.39	23.82
TNUNet-CD	40.48	225.91

变化分析模块（共5层）							信息解码模块		评估指标（IoU）
第1~4层			第5层				信息解码模块		评估指标（IoU）
CCA	Sub	RBIT（32）	CCA	Sub	BIT（4）	RBIT（32）	UNet	UNet++	WHU-CD
	√			√			√		81.13
√			√				√		82.38
√					√		√		76.15
√						√	√		84.47
		√				√		√	83.55
√						√		√	85.44