Self-supervised learning based urban functional zone classification by integrating optical remote sensing image-OSM data

doi:10.11947/j.AGCS.2025.20240067

Abstract

Abstract:

Rapid and accurate classification of urban functional zones (UFZs) provides a scientific basis for urban planning and management and helps to realize sustainable urban development. Although optical remote sensing images provide rich visual information, they cannot fully reflect social attributes and are prone to semantic ambiguity. Therefore, more studies have tried to jointly use data containing urban social attributes (e.g., OSM data) and optical remote sensing images to achieve complementary effects. However, this idea faces two main challenges: first, there are data structure differences between optical images and OSM data, and traditional fusion methods lack sufficient interaction and fusion in the feature extraction stage, which makes it difficult for the model to fully learn the complementary advantages between the data. Second, with the increase of data modalities used for model learning, more manually labeled data are required to train a stable model, but this significantly increases the labor cost of UFZ classification model application. In response to the above problems, this paper proposes a self-supervised learning based urban functional zone classification method by integrating optical remote sensing image-OSM data. On the one hand, OSM data are unified with optical images in terms of spatial distribution and data structure, and then feature extraction and interactive fusion are carried out in a unified multimodal fusion coding architecture to learn cross-modal generalized representations. On the other hand, in this paper, a self-supervised model is used to pre-train on large-scale unlabeled data, and then a small amount of labeled data is used to transfer the model to a specific UFZ classification task, thus reducing the labor cost. The performance advantages of this paper's method over existing mainstream methods are demonstrated by conducting UFZ classification experiments in three large-scale regions, Beijing, Los Angeles and London.

Key words: urban functional zone, optical remote sensing images, OSM, self-supervised learning, multimodality

CLC Number:

P237

Jialing LI, Ji QI, Weipeng LU, Chao TAO. Self-supervised learning based urban functional zone classification by integrating optical remote sensing image-OSM data[J]. Acta Geodaetica et Cartographica Sinica, 2025, 54(1): 154-164.

Figures/Tables 12

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Tab. 1

Tab. 2

Tab. 3

Fig. 7

Fig. 8

Tab. 4

References 32

[1]	杨小军. 城市遥感：城市环境监测、集成与建模[M]. 肖荣波，译. 北京: 高等教育出版社, 2015.
	YANG Xiaojun. Urban remote sensing: monitoring, synthesis and modeling in the urban environment[M]. XIAO Rongbo, trans. Beijing: Higher Education Press, 2015.
[2]	CHEN Lai, KANG Chaogui, YANG chao. Understanding citizens' emotion states under the urban livability environment through social media data: a case study of Wuhan[J]. Journal of Geodesy and Geoinformation Science, 2022, 5(2): 49-59.
[3]	XIAO Zhifeng, LIU Qing, TANG Gefu, et al. Elliptic Fourier transformation-based histograms of oriented gradients for rotationally invariant object detection in remote-sensing images[J]. International Journal of Remote Sensing, 2015, 36(2): 618-644.
[4]	ZHU Qiqi, ZHONG Yanfei, ZHANG Liangpei. Multi-feature probability topic scene classifier for high spatial resolution remote sensing imagery[C]//Proceedings of 2014 IEEE Geoscience and Remote Sensing Symposium. Quebec City: IEEE, 2014: 2854-2857.
[5]	WANG Kaijia, XU Weiming, LI Chuyu, et al. A study on the function and structure of mixed land use in urban built-up areas from the perspective of spatial governance[J]. Journal of Natural Resources, 2023, 38(6): 1496.
[6]	龚健雅, 张觅, 胡翔云, 等. 智能遥感深度学习框架与模型设计[J]. 测绘学报, 2022, 51(4): 475-487. DOI:. doi: 10.11947/j.AGCS.2022.20220027
	GONG Jianya, ZHANG Mi, HU Xiangyun, et al. The design of deep learning framework and model for intelligent remote sensing[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(4): 475-487. DOI:. doi: 10.11947/j.AGCS.2022.20220027
[7]	GUO Zhou, WEN Jiangtian, XU Rui. A shape and size free-CNN for urban functional zone mapping with high-resolution satellite images and POI data[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5622117.
[8]	JOHNSON N, TREIBLE W, CRISPELL D. OpenSentinelMap: a large-scale land use dataset using OpenStreetMap and Sentinel-2 imagery[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New Orleans: IEEE, 2022: 1332-1340.
[9]	LU Weipeng, TAO Chao, LI Haifeng, et al. A unified deep learning framework for urban functional zone extraction based on multi-source heterogeneous data[J]. Remote Sensing of Environment, 2022, 270: 112830.
[10]	GAO Mianxin, GUO Haijing, LIU Longwei, et al. Integrating street view imagery and taxi trajectory for identifying urban function of street space[J/OL]. Geo-spatial Information Science, 2024: 1-23[2023-12-10].https://doi.org/10.1080/10095020.2024.2311866.
[11]	SU Chen, HU Xinli, MENG Qingyan, et al. A multimodal fusion framework for urban scene understanding and functional identification using geospatial data[J]. International Journal of Applied Earth Observation and Geoinformation, 2024, 127: 103696.
[12]	ZHANG Yan, LIU Pengyuan, BILJECKI F. Knowledge and topology: a two layer spatially dependent graph neural networks to identify urban functions with time-series street view image[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2023, 198: 153-168.
[13]	鲁伟鹏, 贺清康, 李佳铃, 等. 结合对象单元和Transformer网络的城市功能区分类[J]. 遥感学报, 2024, 28(8): 1927-1939.
	LU Weipeng, HE Qingkang, LI Jialing, et al. Object units and Transformer networks combined with urban functional zone classification method[J]. National Remote Sensing Bulletin, 2024, 28(8): 1927-1939.
[14]	WU Hao, LUO Wenting, LIN Anqi, et al. SALT: a multifeature ensemble learning framework for mapping urban functional zones from VGI data and VHR images[J]. Computers, Environment and Urban Systems, 2023, 100: 101921.
[15]	DU Shouji, DU Shihong, LIU Bo, et al. Large-scale urban functional zone mapping by integrating remote sensing images and open social data[J]. GIScience & Remote Sensing, 2020, 57(3): 411-430.
[16]	CAO Rui, TU Wei, YANG Cuixin, et al. Deep learning-based remote and social sensing data fusion for urban region function recognition[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 163: 82-97.
[17]	张良培, 何江, 杨倩倩, 等. 数据驱动的多源遥感信息融合研究进展[J]. 测绘学报, 2022, 51(7): 1317-1337. DOI:. doi: 10.11947/j.AGCS.2022.20220171
	ZHANG Liangpei, HE Jiang, YANG Qianqian, et al. Data-driven multi-source remote sensing data fusion: progress and challenges[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(7): 1317-1337. DOI:. doi: 10.11947/j.AGCS.2022.20220171
[18]	XU Shengyu, QING Linbo, HAN Longmei, et al. A new remote sensing images and point-of-interest fused (RPF) model for sensing urban functional regions[J]. Remote Sensing, 2020, 12(6): 1032.
[19]	CHEN Dongsheng, TU Wei, CAO Rui, et al. A hierarchical approach for fine-grained urban villages recognition fusing remote and social sensing data[J]. International Journal of Applied Earth Observation and Geoinformation, 2022, 106: 102661.
[20]	张兵, 杨晓梅, 高连如, 等. 遥感大数据智能解译的地理学认知模型与方法[J]. 测绘学报, 2022, 51(7): 1398-1415. DOI:. doi: 10.11947/j.AGCS.2022.20220279
	ZHANG Bing, YANG Xiaomei, GAO Lianru, et al. Geo-cognitive models and methods for intelligent interpretation of remotely sensed big data[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(7): 1398-1415. DOI:. doi: 10.11947/j.AGCS.2022.20220279
[21]	BAI Lubin, HUANG Weiming, ZHANG Xiuyuan, et al. Geographic mapping with unsupervised multi-modal representation learning from VHR images and POIs[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2023, 201: 193-208.
[22]	ZAKERI F, MARIETHOZ G. A review of geostatistical simulation models applied to satellite remote sensing: methods and applications[J]. Remote Sensing of Environment, 2021, 259: 112381.
[23]	陶超, 阴紫薇, 朱庆, 等. 遥感影像智能解译：从监督学习到自监督学习[J]. 测绘学报, 2021, 50(8): 1122-1134. DOI:. doi: 10.11947/j.AGCS.2021.20210089
	TAO Chao, YIN Ziwei, ZHU Qing, et al. Remote sensing image intelligent interpretation: from supervised learning to self-supervised learning[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8): 1122-1134. DOI:. doi: 10.11947/j.AGCS.2021.20210089
[24]	TAO Chao, QI Ji, GUO Mingning, et al. Self-supervised remote sensing feature learning: learning paradigms, challenges, and future works[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 1-26.
[25]	BACHMANN R, MIZRAHI D, ATANOV A, et al. MultiMAE: multi-modal multi-task masked autoencoders[EB/OL]. [2023-12-10].https://arxiv.org/pdf/2204.01678.
[26]	中国城市规划设计研究院. 城市用地分类与规划建设用地标准：GB 50137—2011[S]. 北京: 中国建筑工业出版社, 2011.
	China Academy of Urban Planning and Design. Code for classification of urban land use and planning standards of development land: GB 50137—2011[S]. Beijing: China Architecture & Building Press, 2011.
[27]	HE Kaiming, CHEN Xinlei, XIE Saining, et al. Masked autoencoders are scalable vision learners[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 15979-15988.
[28]	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[EB/OL]. [2023-12-10].https://arxiv.org/pdf/2010.11929.
[29]	LIU Zhuang, MAO Hanzi, WU Chaoyuan, et al. A ConvNet for the 2020s[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 11966-11976.
[30]	ZHOU Wen, MING Dongping, LÜ Xianwei, et al. SO-CNN based urban functional zone fine division with VHR remote sensing image[J]. Remote Sensing of Environment, 2020, 236: 111458.
[31]	ZHAO Wenzhi, BO Yanchen, CHEN Jiage, et al. Exploring semantic elements for urban scene recognition: deep integration of high-resolution imagery and OpenStreetMap (OSM)[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 151: 237-250.
[32]	GUO Yiwen, TANG Jianbo, LIU Huimin, et al. Identifying up-to-date urban land-use patterns with visual and semantic features based on multisource geospatial data[J]. Sustainable Cities and Society, 2024, 101: 105184.

方法	F₁值							Kappa系数
方法	交通	商业区	居民区	机构区	工地	水体	开放空间	Kappa系数
SO-CNN	0.684	0.310	0.760	0.528	0.238	0.649	0.555	0.514
DIHIO	0.666	0.490	0.748	0.377	0.541	0.732	0.537	0.544
RPF	0.706	0.375	0.766	0.636	0.323	0.619	0.554	0.518
本文方法	0.766	0.515	0.809	0.544	0.603	0.691	0.839	0.623

方法	F₁值								Kappa系数
方法	交通	商业区	居民区	机构区	工地		水体	开放空间	Kappa系数
SO-CNN	0.844	0.507	0.467	0.919	0.315	0.543	0.992	0.887	0.786
DIHIO	0.627	0.736	0.589	0.945	0.630	0.653	0.995	0.856	0.774
RPF	0.783	0.420	0.298	0.866	0.310	0.210	0.939	0.839	0.741
本文方法	0.708	0.810	0.948	0.849	0.790	0.912	0.899	0.932	0.827

方法	F₁值								Kappa系数
方法	交通	商业区	居民区	机构区	工地		水体	开放空间	Kappa系数
SO-CNN	0.482	0.530	0.519	0.843	0.668	0.459	0.801	0.825	0.652
DIHIO	0.397	0.414	0.480	0.842	0.647	0.557	0.695	0.787	0.612
RPF	0.410	0.426	0.464	0.835	0.579	0.366	0.640	0.822	0.613
本文方法	0.772	0.542	0.474	0.933	0.693	0.494	0.889	0.928	0.791

微调输入模态	Kappa系数
微调输入模态	北京数据集	洛杉矶数据集	伦敦数据集
光学	0.609	0.796	0.765
光学+OSM	0.623	0.827	0.791