多尺度特征融合与空间优化的弱监督高分遥感建筑变化检测

doi:10.11947/j.AGCS.2024.20230118

测绘学报 ›› 2024, Vol. 53 ›› Issue (8): 1586-1597.doi: 10.11947/j.AGCS.2024.20230118

多尺度特征融合与空间优化的弱监督高分遥感建筑变化检测

鄢薪¹^,²(), 慎利¹^,²(), 潘俊杰¹^,², 戴延帅¹^,², 王继成³, 郑晓莉⁴, 李志林¹^,²

^1.西南交通大学高速铁路运营安全空间信息技术国家地方联合工程实验室，四川　成都　611756
^2.西南交通大学地球科学与环境工程学院，四川　成都　611756
^3.四川师范大学西南土地资源评价与监测教育部重点实验室，四川　成都　610066
^4.四川省国土科学技术研究院（四川省卫星应用技术中心）耕地资源调查监测与保护利用重点实验室，四川　成都　610045

收稿日期:2023-04-20 发布日期:2024-09-25
通讯作者: 慎利 E-mail:yxecho.swjtu@gmail.com;yxecho.swjtu@gmail.com;lishen@swjtu.edu.cn
作者简介:鄢薪（1995—），男，博士生，研究方向为遥感影像信息提取。E-mail：yxecho.swjtu@gmail.com
基金资助:
国家重点研发计划(2022YFB3904202);国家自然科学基金(42071386);四川省科技厅基本科研业务费项目(2023JDKY0017-3)

Weakly supervised building change detection integrating multi-scale feature fusion and spatial refinement for high resolution remote sensing images

Xin YAN¹^,²(), Li SHEN¹^,²(), Junjie PAN¹^,², Yanshuai DAI¹^,², Jicheng WANG³, Xiaoli ZHENG⁴, Zhi-lin LI¹^,²

^1.State-Province Joint Engineering Laboratory of Spatial Information Technology for High-Speed Railway Safety, Southwest Jiaotong University, Chengdu 611756, China
^2.Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 611756, China
^3.Key Laboratory of Ministry of Education on Land Resources Evaluation and Monitoring in Southwest China, Sichuan Normal University, Chengdu 610066, China
^4.Key Laboratory of Investigation, Monitoring, Protection and Utilization for Cultivated Land Resources, MNR, Sichuan Institute of Land Science and Technology (Sichuan Center of Satellite Application Technology), Chengdu 610045, China

Received:2023-04-20 Published:2024-09-25
Contact: Li SHEN E-mail:yxecho.swjtu@gmail.com;yxecho.swjtu@gmail.com;lishen@swjtu.edu.cn
About author:YAN Xin (1995—), male, PhD candidate, majors in remote sensing image information extraction. E-mail: yxecho.swjtu@gmail.com
Supported by:
The National Key Research and Development Program of China(2022YFB3904202);The National Natural Science Foundation of China(42071386);Science and Technology Project from Department of Natural Resources of Sichuan Province(2023JDKY0017-3)

摘要/Abstract

摘要：

针对建筑物变化检测中深度学习方法严重依赖大量高成本高难度的像素级标注样本进行模型训练的问题，本文提出一种基于图像级标注样本的高分辨率遥感建筑物弱监督变化检测方法MDF-LSR-Net。该方法首先提取双时相多尺度差异特征，并对多尺度差异特征进行渐进式融合，利用充分融合后的多层次多尺度差异特征来生成变化热力图；然后，利用低层融合差异特征的局部空间相似性来优化初始的变化热力图，进一步增强热力图中变化区域的完整性和准确性；最后，基于高质量的变化热力图训练最终的变化检测模型。在公开的建筑物变化检测数据集WHU和LEVIR上的多组试验结果表明，本文方法能够获取更加完整且准确的变化热力图，从而使得基于此训练的变化检测模型也取得更高的检测精度，其中最终的变化检测模型在WHU数据集上的IOU和F₁值分别可达65%和79%以上。

关键词: 高分辨率遥感影像, 建筑物变化检测, 深度学习, 弱监督学习, 多尺度特征融合

Abstract:

To alleviate the heavy dependence of deep learning methods on large-scale high-cost pixel-level annotations, in this paper, we propose a novel weakly supervised method, named MDF-LSR-Net, for high-resolution remote sensing building change detection. Specifically, the proposed method first designs a multi-scale difference feature aggregation module to make better use of multi-scale difference features to generate change heatmaps. Then, by utilizing the local spatial consistency of the low-level fused difference features, MDF-LSR-Net presents a local spatial refinement module to enhance the integrity and accuracy of change regions in heatmaps. Finally, the change detection model is trained based on the high-quality change heatmaps. Experimental results on publicly available datasets, including WHU and LEVIR, demonstrate that our proposed method can obtain more integral and accurate change heatmaps, leading to significantly improved detection performance of the final change detection model. The final model has achieved over 65% points in IOU and over 79% points in F₁ on the WHU dataset.

Key words: high-resolution remote sensing imagery, building change detection, deep learning, weakly supervised learning, multi-scale feature fusion

中图分类号:

P237

鄢薪, 慎利, 潘俊杰, 戴延帅, 王继成, 郑晓莉, 李志林. 多尺度特征融合与空间优化的弱监督高分遥感建筑变化检测[J]. 测绘学报, 2024, 53(8): 1586-1597.

Xin YAN, Li SHEN, Junjie PAN, Yanshuai DAI, Jicheng WANG, Xiaoli ZHENG, Zhi-lin LI. Weakly supervised building change detection integrating multi-scale feature fusion and spatial refinement for high resolution remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(8): 1586-1597.

图/表 8

图1

表1

图2

表2

图3

表3

图4

图5

参考文献 36

[1]	胡明洪, 李佳田, 姚彦吉, 等. 结合多路径的高分辨率遥感影像建筑物提取SER-UNet算法[J]. 测绘学报, 2023, 52(5): 808-817. DOI: 10.11947/j.AGCS.2023.20210691.
	HU Minghong, LI Jiatian, YAO Yanji, et al. SER-UNet algorithm for building extraction from high-resolution remote sensing image combined with multipath[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(5): 808-817. DOI: 10.11947/j.AGCS.2023.20210691.
[2]	刘帅, 李笑迎, 于梦, 等. 高分辨率遥感图像双解耦语义分割网络模型[J]. 测绘学报, 2023, 52(4): 638-647. DOI: 10.11947/j.AGCS.2023.20210455.
	LIU Shuai, LI Xiaoying, YU Meng, et al. Dual decoupling semantic segmentation model for high-resolution remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(4): 638-647. DOI: 10.11947/j.AGCS.2023.20210455.
[3]	乔文凡, 慎利, 戴延帅, 等. 联合膨胀卷积残差网络和金字塔池化表达的高分影像建筑物自动识别[J]. 地理与地理信息科学, 2018, 34(5): 56-62.
	QIAO Wenfan, SHEN Li, DAI Yanshuai, et al. Building extraction from high resolution remote sensing images by combining dilated convolutional residual networks and pyramid pooling representation[J]. Geography and Geo-Information Science, 2018, 34(5): 56-62.
[4]	吴纹辉, 慎利, 董新丰, 等. 面向高分辨率遥感影像建筑物变化检测的边缘感知网络[J]. 地理与地理信息科学, 2021, 37(3): 21-28.
	WU Wenhui, SHEN Li, DONG Xinfeng, et al. Edge sensing network for building change detection in high resolution remote sensing images[J]. Geography and Geo-Information Science, 2021, 37(3): 21-28.
[5]	吴纹辉, 雷添杰. 基于孪生网络和典型语义分割模型的遥感影像变化检测方法框架研究[J]. 国土资源信息化, 2021(2): 15-21.
	WU Wenhui, LEI Tianjie. A framework research of remote sensing image change detection method based on Siamese network and typical semantic segmentation models[J]. Land and Resources Informatization, 2021(2): 15-21.
[6]	LIU Yi, PANG Chao, ZHAN Zongqian, et al. Building change detection for remote sensing images using a dual-task constrained deep Siamese convolutional network model[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(5): 811-815.
[7]	叶沅鑫, 孙苗苗, 周亮, 等. 面向建筑物变化检测的主体边缘分解与重组神经网络[J]. 测绘学报, 2023, 52(1): 71-81. DOI: 10.11947/j.AGCS.2023.20210350.
	YE Yuanxin, SUN Miaomiao, ZHOU Liang, et al. Main body, edge decomposition and reorganization network for building change detection[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(1): 71-81. DOI: 10.11947/j.AGCS.2023.20210350.
[8]	张志强, 张新长, 辛秦川, 等. 结合像元级和目标级的高分辨率遥感影像建筑物变化检测[J]. 测绘学报, 2018, 47(1): 102-112. DOI: 10.11947/j.AGCS.2018.20170483.
	ZHANG Zhiqiang, ZHANG Xinchang, XIN Qinchuan, et al. Combining the pixel-based and object-based methods for building change detection using high-resolution remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(1): 102-112. DOI: 10.11947/j.AGCS.2018.20170483.
[9]	RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]//Proceedings of 2015 International Conference on Medical Image Computing and Computer-assisted intervention. Cham: Springer, 2015: 234-241.
[10]	CHEN L C, ZHU Yukun, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of 2018 European Conference on Computer Vision. Cham: Springer, 2018: 833-851.
[11]	JIANG Huiwei, HU Xiangyun, LI Kun, et al. PGA-SiamNet: pyramid feature-based attention-guided Siamese network for remote sensing orthoimagery building change detection[J]. Remote Sensing, 2020, 12(3): 484.
[12]	SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: visual explanations from deep networks via gradient-based localization[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 618-626.
[13]	MCEVER R A, MANJUNATH B S. PCAMs: weakly supervised semantic segmentation using point supervision[EB/OL]. [2023-02-02]. http://arxiv.org/abs/2007.05615.
[14]	LEE J, YI Jihun, SHIN C, et al. BBAM: bounding box attribution map for weakly supervised semantic and instance segmentation[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 2643-2651.
[15]	ARASLANOV N, ROTH S. Single-stage semantic segmentation from image labels[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020.
[16]	WANG Yude, ZHANG Jie, KAN Meina, et al. Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 12272-12281.
[17]	AHN J, CHO S, KWAK S. Weakly supervised learning of instance segmentation with inter-pixel relations[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 2204-2213.
[18]	李鑫伟, 李彦胜, 张永军. 弱监督深度语义分割网络的多源遥感影像水体检测[J]. 中国图象图形学报, 2021, 26(12): 3015-3026.
	LI Xinwei, LI Yansheng, ZHANG Yongjun. Weakly supervised deep semantic segmentation network for water body extraction based on multi-source remote sensing imagery[J]. Journal of Image and Graphics, 2021, 26(12): 3015-3026.
[19]	YAN Xin, SHEN Li, WANG Jicheng, et al. MSG-SR-Net: a weakly supervised network integrating multiscale generation and superpixel refinement for building extraction from high-resolution remotely sensed imageries[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15: 1012-1023.
[20]	CHEN Jie, HE Fen, ZHANG Yi, et al. SPMF-Net: weakly supervised building segmentation by combining superpixel pooling and multi-scale feature fusion[J]. Remote Sensing, 2020, 12(6): 1049.
[21]	LI Zhenshi, ZHANG Xueliang, XIAO Pengfeng, et al. On the effectiveness of weakly supervised semantic segmentation for building extraction from high-resolution remote sensing imagery[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 3266-3281.
[22]	WEI Yao, JI Shunping. Scribble-based weakly supervised deep learning for road surface extraction from remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5602312.
[23]	LIAN Renbao, HUANG Liqin. Weakly supervised road segmentation in high-resolution remote sensing images using point annotations[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 4501013.
[24]	LU Ming, FANG Leyuan, LI Muxing, et al. NFANet: a novel method for weakly supervised water extraction from high-resolution remote-sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5617114.
[25]	FU Kun, LU Wanxuan, DIAO Wenhui, et al. WSF-NET: weakly supervised feature-fusion network for binary segmentation in remote sensing image[J]. Remote Sensing, 2018, 10(12): 1970.
[26]	KHAN S, HE Xuming, PORIKLI F, et al. Learning deep structured network for weakly supervised change detection[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence. Melbourne: International Joint Conferences on Artificial Intelligence Organization, 2017: 2008-2015.
[27]	ANDERMATT P, TIMOFTE R. A weakly supervised convolutional network for change segmentation and classification[C]//Proceedings of the 15th Asian Conference on Computer Vision. Cham: Springer, 2020: 103-119.
[28]	KALITA I, KARATSIOLIS S, KAMILARIS A. Land use change detection using deep siamese neural networks and weakly supervised learning[C]//Proceedings of the 19th International Conference. Cham: Springer, 2021: 24-35.
[29]	WU Chen, DU Bo, ZHANG Liangpei. Fully convolutional change detection framework with generative adversarial network for unsupervised, weakly supervised and regional supervised change detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(8): 9774-9788.
[30]	HUANG Rui, WANG Ruofei, GUO Qing, et al. Background-mixed augmentation for weakly supervised change detection[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2023, 37(7): 7919-7927.
[31]	王超, 王帅, 陈晓, 等. 联合UNet++和多级差分模块的多源光学遥感影像对象级变化检测[J]. 测绘学报, 2023, 52(2): 283-296. DOI: 10.11947/j.AGCS.2023.20220202.
	WANG Chao, WANG Shuai, CHEN Xiao, et al. Object-level change detection of multi-sensor optical remote sensing images combined with UNet++and multi-level difference module[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(2): 283-296. DOI: 10.11947/j.AGCS.2023.20220202.
[32]	JI Shunping, WEI Shiqing, LU Meng. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(1): 574-586.
[33]	CHEN Hao, SHI Zhenwei. A spatial-temporal attention-based method and a new dataset for remote sensing image change detection[J]. Remote Sensing, 2020, 12(10): 1662.
[34]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[35]	DURAND T, MORDAN T, THOME N, et al. WILDCAT: weakly supervised learning of deep ConvNets for image classification, pointwise localization and segmentation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 5957-5966.
[36]	FAN Junsong, ZHANG Zhaoxiang, TAN Tieniu. Employing multi-estimations for weakly-supervised semantic segmentation[C]//Proceedings 2020 of ECCV. Cham: Springer, 2020: 332-348.

方法	LEVIR数据集				WHU数据集
方法	P	R	F₁值	IOU	P	R	F₁值	IOU
基准方法	38.95	36.16	37.50	23.08	66.27	66.71	66.49	49.80
MDF_base	61.94	60.10	61.01	43.89	87.52	65.84	75.15	60.19
MDF	69.29	71.36	70.31	54.21	83.88	74.38	78.85	65.08
本文方法	69.91	72.35	71.11	55.17	82.91	79.37	81.10	68.21

方法	LEVIR数据集				WHU数据集
方法	P	R	F₁值	IOU	P	R	F₁值	IOU
WILDCAT	31.97	71.03	44.10	28.28	54.42	69.63	61.09	43.98
SEAM	56.89	74.11	64.37	47.46	78.40	73.84	76.05	61.36
WSF－Net	51.51	79.38	62.48	44.65	67.54	81.25	73.76	58.43
MSG－SR－Net	59.28	77.33	67.11	50.50	76.62	77.99	77.30	62.99
本文方法	69.91	72.35	71.11	55.17	82.91	79.37	81.10	68.21

方法	LEVIR数据集				WHU数据集
方法	P	R	F₁值	IOU	P	R	F₁值	IOU
WILDCAT	38.10	46.88	42.04	26.61	56.58	52.91	54.68	37.63
SEAM	49.58	82.77	62.01	44.94	58.56	83.74	68.93	52.59
WSF－Net	48.87	66.61	56.38	39.26	60.53	74.86	66.93	50.30
MSG－SR－Net	52.14	83.98	64.34	47.42	64.60	79.10	71.12	55.18
本文方法	53.08	86.06	65.66	48.88	80.18	78.48	79.32	65.73

多尺度特征融合与空间优化的弱监督高分遥感建筑变化检测

Weakly supervised building change detection integrating multi-scale feature fusion and spatial refinement for high resolution remote sensing images

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 36

相关文章 15

编辑推荐

Metrics

本文评价

[1]	石岩, 王达, 邓敏, 杨学习. 时空异常探测：从数据驱动到知识驱动的内涵转变与实现路径[J]. 测绘学报, 2024, 53(8): 1493-1504.
[2]	徐涛, 杨元维, 高贤君, 王志威, 潘越, 李少华, 许磊, 王艳军, 刘波, 余静, 吴凤敏, 孙浩宇. 融合图卷积与多尺度特征的接触网点云语义分割[J]. 测绘学报, 2024, 53(8): 1624-1633.
[3]	布金伟, 余科根, 汪秋兰, 李玲惠, 刘馨雨, 左小清, 常军. 融合星载GNSS-R数据和多变量参数全球海洋有效波高深度学习反演法[J]. 测绘学报, 2024, 53(7): 1321-1335.
[4]	殷吉崇, 武芳, 翟仁健, 邱越, 巩现勇, 行瑞星. 面向建筑物轮廓规则化的双路径边界约束与相对论生成对抗网络[J]. 测绘学报, 2024, 53(7): 1444-1457.
[5]	江利明, 邵益, 周志伟, 马培峰, 王腾. 智能化InSAR数据处理研究进展、挑战与展望[J]. 测绘学报, 2024, 53(6): 1037-1056.
[6]	郭迟, 刘阳, 罗亚荣, 刘经南, 张全. 图像语义信息在视觉SLAM中的应用研究进展[J]. 测绘学报, 2024, 53(6): 1057-1076.
[7]	龚循强, 汪宏宇, 鲁铁定, 游为. 高铁桥墩沉降的通用渐进分解长期预测网络模型[J]. 测绘学报, 2024, 53(6): 1113-1127.
[8]	顾海燕, 杨懿, 李海涛, 孙立坚, 丁少鹏, 刘世琦. 高分辨率遥感影像样本库动态构建与智能解译应用[J]. 测绘学报, 2024, 53(6): 1165-1179.
[9]	彭代锋, 翟晨晨, 周顶蔚, 张永军, 管海燕, 臧玉府. 基于金字塔语义token全局信息增强的高分光学遥感影像变化检测[J]. 测绘学报, 2024, 53(6): 1195-1211.
[10]	王继成, 郭安嵋, 慎利, 蓝天, 徐柱, 李志林. 多级对比学习下的弱监督高分遥感影像城市固废堆场提取[J]. 测绘学报, 2024, 53(6): 1212-1223.
[11]	丁少鹏, 卢秀山, 刘如飞, 杨懿, 顾海燕, 李海涛. 联合目标特征引导与多重注意力的建筑物变化检测[J]. 测绘学报, 2024, 53(6): 1224-1235.
[12]	刘慧敏, 张陈为, 谌恺祺, 邓敏, 彭翀. 基于深度学习的城市PM2.5浓度时空分布预测及不确定性评估[J]. 测绘学报, 2024, 53(4): 750-760.
[13]	孙传猛, 魏宇, 李欣宇, 马铁华, 武志博. 复杂场景下无水尺水位的影像水位反演智能检测方法[J]. 测绘学报, 2024, 53(3): 558-568.
[14]	廖钊宏, 张依晨, 杨飚, 林明春, 孙文博, 高智. 基于Swin Transformer-CNN的单目遥感影像高程估计方法及其在公路建设场景中的应用[J]. 测绘学报, 2024, 53(2): 344-352.
[15]	林云浩, 王艳军, 李少春, 蔡恒藩. 一种耦合DeepLab与Transformer的农作物种植类型遥感精细分类方法[J]. 测绘学报, 2024, 53(2): 353-366.