多级对比学习下的弱监督高分遥感影像城市固废堆场提取

doi:10.11947/j.AGCS.2024.20230543

测绘学报 ›› 2024, Vol. 53 ›› Issue (6): 1212-1223.doi: 10.11947/j.AGCS.2024.20230543

多级对比学习下的弱监督高分遥感影像城市固废堆场提取

王继成¹(), 郭安嵋²^,³, 慎利²^,³(), 蓝天²^,³, 徐柱²^,³, 李志林²^,³

^1.四川师范大学西南土地资源评价与监测教育部重点实验室，四川　成都　610066
^2.西南交通大学高速铁路运营安全空间信息技术国家地方联合工程实验室，四川　成都　611756
^3.西南交通大学地球科学与环境工程学院，四川　成都　611756

收稿日期:2023-11-23 发布日期:2024-07-22
通讯作者: 慎利 E-mail:wangjicheng11@my.swjtu.edu.cn;lishen@swjtu.edu.cn
作者简介:王继成（1990—），男，博士，研究方向为遥感影像智能分析。　E-mail：wangjicheng11@my.swjtu.edu.cn
基金资助:
国家重点研发计划(2022YFB3904202);国家自然科学基金(42394063);四川省科技计划资助(2023NSFSC19179);四川省科技厅基本科研业务费项目(2023JDKY0017-3)

Multi-level contrastive learning for weakly supervised extraction of urban solid wastes dump from high-resolution remote sensing images

Jicheng WANG¹(), Anmei GUO²^,³, Li SHEN²^,³(), Tian LAN²^,³, Zhu XU²^,³, Zhilin LI²^,³

^1.Key Laboratory of Ministry of Education on Land Resources Evaluation and Monitoring in Southwest China, Sichuan Normal University, Chengdu 610066, China
^2.State-Province Joint Engineering Laboratory of Spatial Information Technology for High-Speed Railway Safety, Southwest Jiaotong University, Chengdu 611756, China
^3.Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 611756, China

Received:2023-11-23 Published:2024-07-22
Contact: Li SHEN E-mail:wangjicheng11@my.swjtu.edu.cn;lishen@swjtu.edu.cn
About author:WANG Jicheng (1990—), male, PhD, majors in remote sensing image intelligent analysis.　E-mail: wangjicheng11@my.swjtu.edu.cn
Supported by:
The National Key Research and Development Program of China(2022YFB3904202);The National Natural Science Foundation of China(42394063);Sichuan Science and Technology Program(2023NSFSC19179);Science and Technology Project from Department of Natural Resources of Sichuan Province(2023JDKY0017-3)

摘要/Abstract

摘要：

城市固体废物是城市化进程中的重要污染源，对城市生态环境和公共健康造成了巨大危害。高分影像固废堆场智能解译是实现自动排查，提升监测效率的核心和关键技术。基于深度学习的固废堆场自动提取方法严重依赖于获取成本高、制作难度大的高质量像素级标注。为此，本文提出使用更易获取的影像级标注，利用影像自监督学习实现像素级固废堆场提取。围绕固废堆场的影像特征，本文方法在尺度对比约束下综合像素、影像两个层次的对比学习方法，对固废堆场的类别激活图细化和完善，并基于此生成高质量的固废堆场伪像素级标注，用于训练固废堆场提取模型。试验结果表明，本文方法在固废堆场提取的F₁值和IoU分数方面分别达到了71.58%和55.74%，显著优于所有对比方法。这说明利用多级对比学习的弱监督方法能够获得更加完整且准确的类别激活图，从而取得更高的固废堆场提取精度。

关键词: 城市固废堆场, 高分辨率遥感影像, 对比学习, 弱监督信息提取, 类别激活图

Abstract:

Urban solid waste is a major pollutant in the urbanization process that endangers the urban ecology and public health. Intelligent interpretation of high-resolution satellite imagery for solid waste dumps (SWD) is crucial for automated monitoring. However, deep learning-based automatic extraction methods for SWD heavily rely on costly and labor-intensive high-quality pixel-level annotations. This paper presents a weakly supervised method that only uses image-level annotations to perform pixel-level SWD extraction. The method leverages the image characteristics of SWD and applies contrastive learning at both pixels, image levels under constraints of scale contrast to improve the class activation maps (CAMs) of SWD. Based on the CAMs, the method generates high-quality pixel-level pseudo-labels that are used to train the SWD extraction model. The experiments on a self-created SWD dataset demonstrate that the proposed method achieves an F₁ score of 71.58% and an IoU score of 55.74%, which are significantly higher than the baseline methods. This shows that the multi-level contrastive learning-based weakly supervised method can produce more complete and accurate CAMs of SWD, leading to better extraction performance.

Key words: urban solid waste dump, high-resolution remote sensing images, contrastive learning, weakly supervised learning, class activation map

中图分类号:

P258

王继成, 郭安嵋, 慎利, 蓝天, 徐柱, 李志林. 多级对比学习下的弱监督高分遥感影像城市固废堆场提取[J]. 测绘学报, 2024, 53(6): 1212-1223.

Jicheng WANG, Anmei GUO, Li SHEN, Tian LAN, Zhu XU, Zhilin LI. Multi-level contrastive learning for weakly supervised extraction of urban solid wastes dump from high-resolution remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(6): 1212-1223.

图/表 9

图1

图2

图3

表1

图4

表2

图5

表3

图6

参考文献 38

[1]	周培疆. 现代环境科学概论[M]. 北京: 科学出版社, 2010.
	ZHOU Peijiang. Introduction to modern environmental science [M]. Beijing: Science Press, 2010.
[2]	李国建, 赵爱华, 张益, 等. 城市垃圾处理工程[J]. 北京: 科学出版社, 2003.
	LI Guojian, ZHAO Aihua, ZHANG Yi, et al. Urban garbage treatment engineering [J]. Beijing: Science Press, 2003.
[3]	彭长琪. 固体废物处理与处置技术[M]. 武汉: 武汉理工大学出版社, 2009.
	PENG Changqi. Solid waste treatment and disposal technology [M]. Wuhan: Wuhan University of Technology Press, 2009.
[4]	赵敬, 臧克, 宫辉力, 等. 遥感技术在北京市垃圾定位及处理中的应用[J]. 首都师范大学学报(自然科学版), 2005, 26(3):109-113.
	ZHAO Jing, ZANG Ke, GONG Huili, et al. The application of remote sensing technology on the distribution and disposal of garbage in Beijing[J]. Journal of Capital Normal University, 2005, 26(3):109-113.
[5]	雒立群, 郭舟, 赵文智, 等. 结合高光谱和高空间分辨率影像提取城市固体废弃物堆[J]. 测绘通报, 2016 (2):38-41, 78.
	LUO Liqun, GUO Zhou, ZHAO Wenzhi, et al. Combining hyperspectral and high-resolution images to extract municipal solid waste dumps[J]. Bulletin of Surveying and Mapping, 2016 (2):38-41, 78.
[6]	张方利, 杜世宏, 郭舟. 应用高分辨率影像的城市固体废弃物提取[J]. 光谱学与光谱分析, 2013, 33(8):2024-2030.
	ZHANG Fangli, DU Shihong, GUO Zhou. Extracting municipal solid waste dumps based on high resolution images[J]. Spectroscopy and Spectral Analysis, 2013, 33(8):2024-2030.
[7]	龚健雅, 张觅, 胡翔云, 等. 智能遥感深度学习框架与模型设计[J]. 测绘学报, 2022, 51(4):475-487.DOI:10.11947/j.AGCS.2022.20220027.
	GONG Jianya, ZHANG Mi, HU Xiangyun, et al. The design of deep learning framework and model for intelligent remote sensing[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(4):475-487.DOI:10.11947/j.AGCS.2022.20220027.
[8]	陈军, 刘万增, 武昊, 等. 智能化测绘的基本问题与发展方向[J]. 测绘学报, 2021, 50(8):995-1005.DOI:10.11947/j.AGCS.2021.20210235.
	CHEN Jun, LIU Wanzeng, WU Hao, et al. Smart surveying and mapping: fundamental issues and research agenda[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):995-1005.DOI:10.11947/j.AGCS.2021.20210235.
[9]	TORRES R N, FRATERNALI P. Learning to identify illegal landfills through scene classification in aerial images[J]. Remote Sensing, 2021, 13(22):4520.
[10]	YOUME O, BAYET T, DEMBELE J M, et al. Deep learning and remote sensing: detection of dumping waste using UAV[J]. Procedia Computer Science, 2021, 185:361-369.
[11]	LI Huifang, HU Chao, ZHONG Xinrun, et al. Solid waste detection in cities using remote sensing imagery based on a location-guided key point network with multiple enhancements[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023, 16:191-201.
[12]	NIU Bowen, FENG Quanlong, YANG Jianyu, et al. Solid waste mapping based on very high resolution remote sensing imagery and a novel deep learning approach[J]. Geocarto International, 2023, 38(1):2164361.
[13]	王浩. 融入图卷积全局推理的固废堆场高分遥感检测及其环境影响力评估[D]. 成都: 西南交通大学, 2021.
	WANG Hao. Remote sensing detection of high score in solid waste yard and its environmental impact assessment based on global reasoning of graph volume product[D]. Chengdu: Southwest Jiaotong University, 2021.
[14]	张蜀军. 基于实例分割的高分遥感无序固废堆场识别[D]. 成都: 西南交通大学, 2021.
	ZHANG Shujun. Disorderd solid waste yards recognition from high-resolution remote sensing images based on instance segmentation[D]. Chengdu: Southwest Jiaotong University, 2021.
[15]	TORRES R N, FRATERNALI P. Aerial waste dataset for landfill discovery in aerial and satellite images[J]. Scientific Data, 2023, 10:63.
[16]	季顺平, 魏世清. 遥感影像建筑物提取的卷积神经元网络与开源数据集方法[J]. 测绘学报, 2019, 48(4):448-459.DOI:10.11947/j.AGCS.2019.20180206.
	JI Shunping, WEI Shiqing. Building extraction via convolutional neural networks from an open remote sensing building dataset[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(4):448-459.DOI:10.11947/j.AGCS.2019.20180206.
[17]	龚健雅, 许越, 胡翔云, 等. 遥感影像智能解译样本库现状与研究[J]. 测绘学报, 2021, 50(8):1013-1022.DOI:10.11947/j.AGCS.2021.20210085.
	GONG Jianya, XU Yue, HU Xiangyun, et al. Status analysis and research of sample database for intelligent interpretation of remote sensing image[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):1013-1022.DOI:10.11947/j.AGCS.2021.20210085.
[18]	陶超, 阴紫薇, 朱庆, 等. 遥感影像智能解译：从监督学习到自监督学习[J]. 测绘学报, 2021, 50(8):1122-1134.DOI:10.11947/j.AGCS.2021.20210089.
	TAO Chao, YIN Ziwei, ZHU Qing, et al. Remote sensing image intelligent interpretation: from supervised learning to self-supervised learning[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):1122-1134.DOI:10.11947/j.AGCS.2021.20210089.
[19]	张继贤, 李海涛, 顾海燕, 等. 人机协同的自然资源要素智能提取方法[J]. 测绘学报, 2021, 50(8):1023-1032.DOI:10.11947/j.AGCS.2021.20210102.
	ZHANG Jixian, LI Haitao, GU Haiyan, et al. Study on man-machine collaborative intelligent extraction for natural resource features[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):1023-1032.DOI:10.11947/j.AGCS.2021.20210102.
[20]	陈军, 艾廷华, 闫利, 等. 智能化测绘的混合计算范式与方法研究[J/OL]. 测绘学报: 1-19 [2024-06-11].http://kns.cnki.net/kcms/detail/11.2089.P.20240415.1049.002.html.
	CHEN Jun, AI Tinghua, YAN li, et al. Hybrid computational paradigm and methods for intelligentized surveying and mapping[J/OL]. Acta Geodaetica et Cartographica Sinica: 1-19 [2024-06-11].http://kns.cnki.net/kcms/detail/11.2089.P.20240415.1049.002.html.
[21]	WANG Yude, ZHANG Jie, KAN Meina, et al. Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 12275-12284.
[22]	WANG Jicheng, YAN Xin, SHEN Li, et al. Scale-invariant multi-level context aggregation network for weakly supervised building extraction[J]. Remote Sensing, 2023, 15(5):1432.
[23]	CHAN L, HOSSEINI M S, PLATANIOTIS K N. A comprehensive analysis of weakly-supervised semantic segmentation in different image domains[J]. International Journal of Computer Vision, 2021, 129(2):361-384.
[24]	ZHOU Bolei, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discriminative localization[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 2921-2929.
[25]	CAO Yinxia, HUANG Xin. A coarse-to-fine weakly supervised learning method for green plastic cover segmentation using high-resolution remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 188:157-176.
[26]	FU Kun, LU Wanxuan, DIAO Wenhui, et al. WSF-NET: weakly supervised feature-fusion network for binary segmentation in remote sensing image[J]. Remote Sensing, 2018, 10(12):1970.
[27]	YAN Xin, SHEN Li, WANG Jicheng, et al. MSG-SR-net: a weakly supervised network integrating multiscale generation and superpixel refinement for building extraction from high-resolution remotely sensed imageries[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15:1012-1023.
[28]	LI Zhenshi, ZHANG Xueliang, XIAO Pengfeng, et al. On the effectiveness of weakly supervised semantic segmentation for building extraction from high-resolution remote sensing imagery[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14:3266-3281.
[29]	CHEN Jie, HE Fen, ZHANG Yi, et al. SPMF-net: weakly supervised building segmentation by combining superpixel pooling and multi-scale feature fusion[J]. Remote Sensing, 2020, 12(6):1049.
[30]	FANG Fang, ZHENG Daoyuan, LI Shengwen, et al. Improved pseudomasks generation for weakly supervised building extraction from high-resolution remote sensing imagery[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15:1629-1642.
[31]	YAN Xin, SHEN Li, WANG Jicheng, et al. PANet: pixelwise affinity network for weakly supervised building extraction from high-resolution remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19:1-5.
[32]	鄢薪, 慎利, 潘俊杰, 等. 多尺度特征融合与空间优化的弱监督高分遥感建筑变化检测[J/OL]. 测绘学报: 1-14 [2024-02-17].http://kns.cnki.net/kcms/detail/11.2089.P.20231109.1026.002.html.
	YAN Xin, SHEN Li, PAN Junjie, et al. Weakly supervised building change detection integrating multi-scale feature fusion and spatial refinement for high resolution remote sensing images[J/OL]. Acta Geodaetica et Cartographica Sinica: 1-14 [2024-02-17].http://kns.cnki.net/kcms/detail/11.2089.P.20231109.1026.002.html.
[33]	ZHOU Tianfei, ZHANG Meijie, ZHAO Fang, et al. Regional semantic contrast and aggregation for weakly supervised semantic segmentation[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 4299-4309.
[34]	YOON S H, KWEON H, JEONG J, et al. Exploring pixel-level self-supervision for weakly supervised semantic segmentation[EB/OL]. [2024-01-25]. https://arxiv.org/pdf/2112.05351.pdf.
[35]	KHOSLA P, TETERWAK P, WANG Chen, et al. Supervised contrastive learning[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver: ACM Press, 2020: 18661-18673.
[36]	WANG Wenguan, SUN Guolei, VAN GOOL L. Looking beyond single images for weakly supervised semantic segmentation learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(3):1635-1649.
[37]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[38]	CHATTOPADHAY A, SARKAR A, HOWLADER P, et al. Grad-CAM++: generalized gradient-based visual explanations for deep convolutional networks[C]//Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision. Lake Tahoe: IEEE, 2018: 839-847.

方法	精确率	召回率	F₁值	IOU
OriginCAM	34.16	40.95	37.49	23.07
ScaleNet	44.76	43.16	43.95	28.16
Scene ScaleNet	50.34	53.11	51.69	34.85
Pixel ScaleNet	60.23	62.33	61.26	44.16
SPScaleNet	62.91	64.35	63.62	46.65

方法	精确率	召回率	F₁值	IoU
GradCAM++	41.73	52.34	46.44	30.24
MCIS	15.26	30.14	20.26	11.27
SEAM	45.88	60.14	52.05	35.18
PANet	57.39	65.38	61.13	44.01
WSFNet	43.17	59.83	50.15	33.47
SPScaleNet	62.91	64.35	63.62	46.65

方法	精确率	召回率	F₁值	IoU
WSF-Net	45.56	62.44	52.68	35.76
SEAM	49.47	60.14	54.29	37.25
PANet	64.21	69.14	66.58	49.91
SPScaleNet	68.90	74.48	71.58	55.74

多级对比学习下的弱监督高分遥感影像城市固废堆场提取

Multi-level contrastive learning for weakly supervised extraction of urban solid wastes dump from high-resolution remote sensing images

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 38

相关文章 15

编辑推荐

Metrics

本文评价

[1]	殷吉崇, 武芳, 翟仁健, 邱越, 巩现勇, 行瑞星. 面向建筑物轮廓规则化的双路径边界约束与相对论生成对抗网络[J]. 测绘学报, 2024, 53(7): 1444-1457.
[2]	顾海燕, 杨懿, 李海涛, 孙立坚, 丁少鹏, 刘世琦. 高分辨率遥感影像样本库动态构建与智能解译应用[J]. 测绘学报, 2024, 53(6): 1165-1179.
[3]	彭代锋, 翟晨晨, 周顶蔚, 张永军, 管海燕, 臧玉府. 基于金字塔语义token全局信息增强的高分光学遥感影像变化检测[J]. 测绘学报, 2024, 53(6): 1195-1211.
[4]	林云浩, 王艳军, 李少春, 蔡恒藩. 一种耦合DeepLab与Transformer的农作物种植类型遥感精细分类方法[J]. 测绘学报, 2024, 53(2): 353-366.
[5]	胡明洪, 李佳田, 姚彦吉, 阿晓荟, 陆美, 李文. 结合多路径的高分辨率遥感影像建筑物提取SER-UNet算法[J]. 测绘学报, 2023, 52(5): 808-817.
[6]	胡安娜, 刘睿, 吴亮, 张进, 徐永洋, 陈思琼. 顾及全局特征和纹理特征的遥感影像超分辨率重建方法[J]. 测绘学报, 2023, 52(4): 648-659.
[7]	姜明, 张新长, 孙颖, 冯炜明, 阮永俭. 全尺度特征聚合的高分辨率遥感影像变化检测网络[J]. 测绘学报, 2023, 52(10): 1738-1748.
[8]	张广斌, 高贤君, 冉树浩, 杨元维, 李丽珊, 张妍. 高分遥感影像云雪共存区轻量云高精度检测方法[J]. 测绘学报, 2023, 52(1): 93-107.
[9]	何直蒙, 丁海勇, 安炳琪. 高分辨率遥感影像建筑物提取的空洞卷积E-Unet算法[J]. 测绘学报, 2022, 51(3): 457-467.
[10]	张玉鑫, 颜青松, 邓非. 高分辨率遥感影像建筑物提取多路径RSU网络法[J]. 测绘学报, 2022, 51(1): 135-144.
[11]	张涛, 丁乐乐, 史芙蓉. 高分辨率遥感影像城中村提取的景观语义指数方法[J]. 测绘学报, 2021, 50(1): 97-104.
[12]	左宗成, 张文, 张东映. 融合可变形卷积与条件随机场的遥感影像语义分割方法[J]. 测绘学报, 2019, 48(6): 718-726.
[13]	董志鹏, 王密, 李德仁, 王艳丽, 张致齐. 遥感影像目标的尺度特征卷积神经网络识别法[J]. 测绘学报, 2019, 48(10): 1285-1295.
[14]	范荣双, 陈洋, 徐启恒, 王竞雪. 基于深度学习的高分辨率遥感影像建筑物提取方法[J]. 测绘学报, 2019, 48(1): 34-41.
[15]	董志鹏, 王密, 李德仁, 王艳丽, 张致齐. 利用对象光谱与纹理实现高分辨率遥感影像云检测方法[J]. 测绘学报, 2018, 47(7): 996-1006.