Multi-level contrastive learning for weakly supervised extraction of urban solid wastes dump from high-resolution remote sensing images

doi:10.11947/j.AGCS.2024.20230543

Abstract

Abstract:

Urban solid waste is a major pollutant in the urbanization process that endangers the urban ecology and public health. Intelligent interpretation of high-resolution satellite imagery for solid waste dumps (SWD) is crucial for automated monitoring. However, deep learning-based automatic extraction methods for SWD heavily rely on costly and labor-intensive high-quality pixel-level annotations. This paper presents a weakly supervised method that only uses image-level annotations to perform pixel-level SWD extraction. The method leverages the image characteristics of SWD and applies contrastive learning at both pixels, image levels under constraints of scale contrast to improve the class activation maps (CAMs) of SWD. Based on the CAMs, the method generates high-quality pixel-level pseudo-labels that are used to train the SWD extraction model. The experiments on a self-created SWD dataset demonstrate that the proposed method achieves an F₁ score of 71.58% and an IoU score of 55.74%, which are significantly higher than the baseline methods. This shows that the multi-level contrastive learning-based weakly supervised method can produce more complete and accurate CAMs of SWD, leading to better extraction performance.

Key words: urban solid waste dump, high-resolution remote sensing images, contrastive learning, weakly supervised learning, class activation map

CLC Number:

P258

Jicheng WANG, Anmei GUO, Li SHEN, Tian LAN, Zhu XU, Zhilin LI. Multi-level contrastive learning for weakly supervised extraction of urban solid wastes dump from high-resolution remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(6): 1212-1223.

Figures/Tables 9

Fig.1

Fig.2

Fig.3

Tab.1

Fig.4

Tab.2

Fig.5

Tab.3

Fig.6

References 38

[1]	周培疆. 现代环境科学概论[M]. 北京: 科学出版社, 2010.
	ZHOU Peijiang. Introduction to modern environmental science [M]. Beijing: Science Press, 2010.
[2]	李国建, 赵爱华, 张益, 等. 城市垃圾处理工程[J]. 北京: 科学出版社, 2003.
	LI Guojian, ZHAO Aihua, ZHANG Yi, et al. Urban garbage treatment engineering [J]. Beijing: Science Press, 2003.
[3]	彭长琪. 固体废物处理与处置技术[M]. 武汉: 武汉理工大学出版社, 2009.
	PENG Changqi. Solid waste treatment and disposal technology [M]. Wuhan: Wuhan University of Technology Press, 2009.
[4]	赵敬, 臧克, 宫辉力, 等. 遥感技术在北京市垃圾定位及处理中的应用[J]. 首都师范大学学报(自然科学版), 2005, 26(3):109-113.
	ZHAO Jing, ZANG Ke, GONG Huili, et al. The application of remote sensing technology on the distribution and disposal of garbage in Beijing[J]. Journal of Capital Normal University, 2005, 26(3):109-113.
[5]	雒立群, 郭舟, 赵文智, 等. 结合高光谱和高空间分辨率影像提取城市固体废弃物堆[J]. 测绘通报, 2016 (2):38-41, 78.
	LUO Liqun, GUO Zhou, ZHAO Wenzhi, et al. Combining hyperspectral and high-resolution images to extract municipal solid waste dumps[J]. Bulletin of Surveying and Mapping, 2016 (2):38-41, 78.
[6]	张方利, 杜世宏, 郭舟. 应用高分辨率影像的城市固体废弃物提取[J]. 光谱学与光谱分析, 2013, 33(8):2024-2030.
	ZHANG Fangli, DU Shihong, GUO Zhou. Extracting municipal solid waste dumps based on high resolution images[J]. Spectroscopy and Spectral Analysis, 2013, 33(8):2024-2030.
[7]	龚健雅, 张觅, 胡翔云, 等. 智能遥感深度学习框架与模型设计[J]. 测绘学报, 2022, 51(4):475-487.DOI:10.11947/j.AGCS.2022.20220027.
	GONG Jianya, ZHANG Mi, HU Xiangyun, et al. The design of deep learning framework and model for intelligent remote sensing[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(4):475-487.DOI:10.11947/j.AGCS.2022.20220027.
[8]	陈军, 刘万增, 武昊, 等. 智能化测绘的基本问题与发展方向[J]. 测绘学报, 2021, 50(8):995-1005.DOI:10.11947/j.AGCS.2021.20210235.
	CHEN Jun, LIU Wanzeng, WU Hao, et al. Smart surveying and mapping: fundamental issues and research agenda[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):995-1005.DOI:10.11947/j.AGCS.2021.20210235.
[9]	TORRES R N, FRATERNALI P. Learning to identify illegal landfills through scene classification in aerial images[J]. Remote Sensing, 2021, 13(22):4520.
[10]	YOUME O, BAYET T, DEMBELE J M, et al. Deep learning and remote sensing: detection of dumping waste using UAV[J]. Procedia Computer Science, 2021, 185:361-369.
[11]	LI Huifang, HU Chao, ZHONG Xinrun, et al. Solid waste detection in cities using remote sensing imagery based on a location-guided key point network with multiple enhancements[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023, 16:191-201.
[12]	NIU Bowen, FENG Quanlong, YANG Jianyu, et al. Solid waste mapping based on very high resolution remote sensing imagery and a novel deep learning approach[J]. Geocarto International, 2023, 38(1):2164361.
[13]	王浩. 融入图卷积全局推理的固废堆场高分遥感检测及其环境影响力评估[D]. 成都: 西南交通大学, 2021.
	WANG Hao. Remote sensing detection of high score in solid waste yard and its environmental impact assessment based on global reasoning of graph volume product[D]. Chengdu: Southwest Jiaotong University, 2021.
[14]	张蜀军. 基于实例分割的高分遥感无序固废堆场识别[D]. 成都: 西南交通大学, 2021.
	ZHANG Shujun. Disorderd solid waste yards recognition from high-resolution remote sensing images based on instance segmentation[D]. Chengdu: Southwest Jiaotong University, 2021.
[15]	TORRES R N, FRATERNALI P. Aerial waste dataset for landfill discovery in aerial and satellite images[J]. Scientific Data, 2023, 10:63.
[16]	季顺平, 魏世清. 遥感影像建筑物提取的卷积神经元网络与开源数据集方法[J]. 测绘学报, 2019, 48(4):448-459.DOI:10.11947/j.AGCS.2019.20180206.
	JI Shunping, WEI Shiqing. Building extraction via convolutional neural networks from an open remote sensing building dataset[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(4):448-459.DOI:10.11947/j.AGCS.2019.20180206.
[17]	龚健雅, 许越, 胡翔云, 等. 遥感影像智能解译样本库现状与研究[J]. 测绘学报, 2021, 50(8):1013-1022.DOI:10.11947/j.AGCS.2021.20210085.
	GONG Jianya, XU Yue, HU Xiangyun, et al. Status analysis and research of sample database for intelligent interpretation of remote sensing image[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):1013-1022.DOI:10.11947/j.AGCS.2021.20210085.
[18]	陶超, 阴紫薇, 朱庆, 等. 遥感影像智能解译：从监督学习到自监督学习[J]. 测绘学报, 2021, 50(8):1122-1134.DOI:10.11947/j.AGCS.2021.20210089.
	TAO Chao, YIN Ziwei, ZHU Qing, et al. Remote sensing image intelligent interpretation: from supervised learning to self-supervised learning[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):1122-1134.DOI:10.11947/j.AGCS.2021.20210089.
[19]	张继贤, 李海涛, 顾海燕, 等. 人机协同的自然资源要素智能提取方法[J]. 测绘学报, 2021, 50(8):1023-1032.DOI:10.11947/j.AGCS.2021.20210102.
	ZHANG Jixian, LI Haitao, GU Haiyan, et al. Study on man-machine collaborative intelligent extraction for natural resource features[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):1023-1032.DOI:10.11947/j.AGCS.2021.20210102.
[20]	陈军, 艾廷华, 闫利, 等. 智能化测绘的混合计算范式与方法研究[J/OL]. 测绘学报: 1-19 [2024-06-11].http://kns.cnki.net/kcms/detail/11.2089.P.20240415.1049.002.html.
	CHEN Jun, AI Tinghua, YAN li, et al. Hybrid computational paradigm and methods for intelligentized surveying and mapping[J/OL]. Acta Geodaetica et Cartographica Sinica: 1-19 [2024-06-11].http://kns.cnki.net/kcms/detail/11.2089.P.20240415.1049.002.html.
[21]	WANG Yude, ZHANG Jie, KAN Meina, et al. Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 12275-12284.
[22]	WANG Jicheng, YAN Xin, SHEN Li, et al. Scale-invariant multi-level context aggregation network for weakly supervised building extraction[J]. Remote Sensing, 2023, 15(5):1432.
[23]	CHAN L, HOSSEINI M S, PLATANIOTIS K N. A comprehensive analysis of weakly-supervised semantic segmentation in different image domains[J]. International Journal of Computer Vision, 2021, 129(2):361-384.
[24]	ZHOU Bolei, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discriminative localization[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 2921-2929.
[25]	CAO Yinxia, HUANG Xin. A coarse-to-fine weakly supervised learning method for green plastic cover segmentation using high-resolution remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 188:157-176.
[26]	FU Kun, LU Wanxuan, DIAO Wenhui, et al. WSF-NET: weakly supervised feature-fusion network for binary segmentation in remote sensing image[J]. Remote Sensing, 2018, 10(12):1970.
[27]	YAN Xin, SHEN Li, WANG Jicheng, et al. MSG-SR-net: a weakly supervised network integrating multiscale generation and superpixel refinement for building extraction from high-resolution remotely sensed imageries[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15:1012-1023.
[28]	LI Zhenshi, ZHANG Xueliang, XIAO Pengfeng, et al. On the effectiveness of weakly supervised semantic segmentation for building extraction from high-resolution remote sensing imagery[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14:3266-3281.
[29]	CHEN Jie, HE Fen, ZHANG Yi, et al. SPMF-net: weakly supervised building segmentation by combining superpixel pooling and multi-scale feature fusion[J]. Remote Sensing, 2020, 12(6):1049.
[30]	FANG Fang, ZHENG Daoyuan, LI Shengwen, et al. Improved pseudomasks generation for weakly supervised building extraction from high-resolution remote sensing imagery[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15:1629-1642.
[31]	YAN Xin, SHEN Li, WANG Jicheng, et al. PANet: pixelwise affinity network for weakly supervised building extraction from high-resolution remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19:1-5.
[32]	鄢薪, 慎利, 潘俊杰, 等. 多尺度特征融合与空间优化的弱监督高分遥感建筑变化检测[J/OL]. 测绘学报: 1-14 [2024-02-17].http://kns.cnki.net/kcms/detail/11.2089.P.20231109.1026.002.html.
	YAN Xin, SHEN Li, PAN Junjie, et al. Weakly supervised building change detection integrating multi-scale feature fusion and spatial refinement for high resolution remote sensing images[J/OL]. Acta Geodaetica et Cartographica Sinica: 1-14 [2024-02-17].http://kns.cnki.net/kcms/detail/11.2089.P.20231109.1026.002.html.
[33]	ZHOU Tianfei, ZHANG Meijie, ZHAO Fang, et al. Regional semantic contrast and aggregation for weakly supervised semantic segmentation[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 4299-4309.
[34]	YOON S H, KWEON H, JEONG J, et al. Exploring pixel-level self-supervision for weakly supervised semantic segmentation[EB/OL]. [2024-01-25]. https://arxiv.org/pdf/2112.05351.pdf.
[35]	KHOSLA P, TETERWAK P, WANG Chen, et al. Supervised contrastive learning[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver: ACM Press, 2020: 18661-18673.
[36]	WANG Wenguan, SUN Guolei, VAN GOOL L. Looking beyond single images for weakly supervised semantic segmentation learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(3):1635-1649.
[37]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[38]	CHATTOPADHAY A, SARKAR A, HOWLADER P, et al. Grad-CAM++: generalized gradient-based visual explanations for deep convolutional networks[C]//Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision. Lake Tahoe: IEEE, 2018: 839-847.

方法	精确率	召回率	F₁值	IOU
OriginCAM	34.16	40.95	37.49	23.07
ScaleNet	44.76	43.16	43.95	28.16
Scene ScaleNet	50.34	53.11	51.69	34.85
Pixel ScaleNet	60.23	62.33	61.26	44.16
SPScaleNet	62.91	64.35	63.62	46.65

方法	精确率	召回率	F₁值	IoU
GradCAM++	41.73	52.34	46.44	30.24
MCIS	15.26	30.14	20.26	11.27
SEAM	45.88	60.14	52.05	35.18
PANet	57.39	65.38	61.13	44.01
WSFNet	43.17	59.83	50.15	33.47
SPScaleNet	62.91	64.35	63.62	46.65

方法	精确率	召回率	F₁值	IoU
WSF-Net	45.56	62.44	52.68	35.76
SEAM	49.47	60.14	54.29	37.25
PANet	64.21	69.14	66.58	49.91
SPScaleNet	68.90	74.48	71.58	55.74