Object-oriented high-resolution image classification using inductive graph neural networks

doi:10.11947/j.AGCS.2024.20230224

Abstract

Abstract:

Traditional object-oriented classification methods mostly use spectral features of image objects and ignore the spatial features among image objects. In this paper, an object-oriented classification method for high-resolution remote sensing images using improved inductive graph neural network is proposed. The method is able to adaptively adjust the fusion coefficient of spectral-spatial composite node similarity and automatically determine the optimal sampling number of neighboring nodes. First, we improved the K-nearest neighbor (KNN) graph construction method. The standard deviation informativeness evaluation method was used to determine the fusion coefficients for constructing the composite node similarity of spectral and spatial features. Then, the optimal sampling number of neighboring nodes was determined using the feedback curve method, and feature representation was accomplished using GraphSAGE node embedding. Finally, the classifications of the nodes were predicted by Softmax function. We used GID-15 and BDCI2017 datasets as experimental data. The proposed graph construction method has improved the classification accuracy. The average Kappa coefficient of the proposed method was better than CART, GCN, GAT, LANet, CCTNet, and SLCNet by 0.31, 0.14, 0.13, 0.12, 0.08, and 0.02. The average overall accuracy, on the other hand, was better than 42.31%, 7.4%, 6.73%, 8.69%, 6.03%, and 1.52%. Meanwhile, our method had good robustness in vegetation and built-up land extraction. The method proposed in this paper provides an effective tool for land cover classification of high-resolution remote sensing images.

Key words: high-resolution remote sensing images, GraphSAGE, node connection weights, aggregation nodes, land cover classification

CLC Number:

P237

Zhiwei XIE, Shuaizhi ZHAI, Fengyuan ZHANG, Min CHEN, Lishuang SUN. Object-oriented high-resolution image classification using inductive graph neural networks[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(8): 1610-1623.

Figures/Tables 16

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Tab.1

Tab.2

Tab.3

Fig.9

Fig.10

Fig.11

Tab.4

References 46

[1]	王密, 杨芳. 智能遥感卫星与遥感影像实时服务[J]. 测绘学报, 2019, 48(12): 1586-1594. DOI: 10.11947/j.AGCS.2019.20190454.
	WANG Mi, YANG Fang. Intelligent remote sensing satellite and remote sensing image real-time service[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(12): 1586-1594. DOI: 10.11947/j.AGCS.2019.20190454.
[2]	ZHU Qiqi, GUO Xi, DENG Weihuan, et al. Land-use/land-cover change detection based on a Siamese global learning framework for high spatial resolution remote sensing imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 184: 63-78.
[3]	周培诚, 程塨, 姚西文, 等. 高分辨率遥感影像解译中的机器学习范式[J]. 遥感学报, 2021, 25(1): 182-197.
	ZHOU Peicheng, CHENG Gong, YAO Xiwen, et al. Machine learning paradigms in high-resolution remote sensing image interpretation[J]. National Remote Sensing Bulletin, 2021, 25(1): 182-197.
[4]	KAUR S, BANSAL R K, MITTAL M, et al. Mixed pixel decomposition based on extended fuzzy clustering for single spectral value remote sensing images[J]. Journal of the Indian Society of Remote Sensing, 2019, 47(3): 427-437.
[5]	王志盼, 沈彦, 王亮, 等. 单类分类框架下的高分辨率遥感影像建筑物变化检测算法[J]. 武汉大学学报(信息科学版), 2020, 45(10): 1610-1618.
	WANG Zhipan, SHEN Yan, WANG Liang, et al. High-resolution remote sensing image building change detection based on one-class classifier framework[J]. Geomatics and Information Science of Wuhan University, 2020, 45(10): 1610-1618.
[6]	LI Zhiqing, LI Erzhu, SAMAT A, et al. An object-oriented CNN model based on improved superpixel segmentation for high-resolution remote sensing image classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15: 4782-4796.
[7]	DOU Jie, CHANG K T, CHEN Shuisen, et al. Automatic case-based reasoning approach for landslide detection: integration of object-oriented image analysis and a genetic algorithm[J]. Remote Sensing, 2015, 7(4): 4318-4342.
[8]	KATTENBORN T, LEITLOFF J, SCHIEFER F, et al. Review on convolutional neural networks (CNN) in vegetation remote sensing[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 173: 24-49.
[9]	GONG Jianya, JI Shunping. Photogrammetry and deep learning[J]. Journal of Geodesy and Geoinformation Science, 2018, 1(1): 1-15.
[10]	PAN Deng, ZHANG Meng, ZHANG Bo. A generic FCN-based approach for the road-network extraction from VHR remote sensing images-using OpenStreetMap as benchmarks[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 2662-2673.
[11]	HUANG Wei, TANG Hong, XU Penglei. OEC-RNN: object-oriented delineation of rooftops with edges and corners using the recurrent neural network from the aerial images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-12.
[12]	AUDEBERT N, LE S B, LEFÈVRE S. Beyond RGB: very high resolution urban remote sensing with multimodal deep networks[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 140: 20-32.
[13]	HONG Danfeng, GAO Lianru, YOKOYA N, et al. More diverse means better: multimodal deep learning meets remote-sensing imagery classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59(5): 4340-4354.
[14]	ROY S K, DERIA A, HONG Danfeng, et al. Multimodal fusion transformer for remote sensing image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 1-20.
[15]	靳志宾, 蒲英霞, 陈刚, 等. 基于地理加权的k-NN高分辨率遥感影像分类算法改进[J]. 遥感技术与应用, 2013, 28(1): 97-102.
	JIN Zhibin, PU Yingxia, CHEN Gang, et al. The modified k-NN classifier for high spatial resolution remote sensing images based on geographical weighted models[J]. Remote Sensing Technology and Application, 2013, 28(1): 97-102.
[16]	洪亮, 冯亚飞, 彭双云, 等. 面向对象的多尺度加权联合稀疏表示的高空间分辨率遥感影像分类[J]. 测绘学报, 2022, 51(2): 224-237. DOI: 10.11947/j.AGCS.2022.20190290.
	HONG Liang, FENG Yafei, PENG Shuangyun, et al. Classification of high spatial resolution remote sensing imagery based on object-oriented multi-scale weighted sparse representation[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(2): 224-237. DOI: 10.11947/j.AGCS.2022.20190290.
[17]	YUAN Qiangqiang, SHEN Huanfeng, LI Tongwen, et al. Deep learning in environmental remote sensing: achievements and challenges[J]. Remote Sensing of Environment, 2020, 241: 111716.
[18]	CHENG Gong, XIE Xingxing, HAN Junwei, et al. Remote sensing image scene classification meets deep learning: challenges, methods, benchmarks, and opportunities[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 3735-3756.
[19]	陈云浩, 冯通, 史培军, 等. 基于面向对象和规则的遥感影像分类研究[J]. 武汉大学学报(信息科学版), 2006, 31(4): 316-320.
	CHEN Yunhao, FENG Tong, SHI Peijun, et al. Classification of remote sensing image based on object oriented and class rules[J]. Geomatics and Information Science of Wuhan University, 2006, 31(4): 316-320.
[20]	薄树奎, 韩新超, 丁琳. 面向对象影像分类中分割参数的选择[J]. 武汉大学学报(信息科学版), 2009, 34(5): 514-517.
	BO Shukui, HAN Xinchao, DING Lin. Automatic selection of segmentation parameters for object oriented image classification[J]. Geomatics and Information Science of Wuhan University, 2009, 34(5): 514-517.
[21]	SAHA S, ZHAO Shan, ZHU Xiaoxiang. Multitarget domain adaptation for remote sensing classification using graph neural network[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 1-5.
[22]	ZHOU Jie, CUI Ganqu, HU Shengding, et al. Graph neural networks: a review of methods and applications[J]. AI Open, 2020, 1: 57-81.
[23]	吴博, 梁循, 张树森, 等. 图神经网络前沿进展与应用[J]. 计算机学报, 2022, 45(1): 35-68.
	WU Bo, LIANG Xun, ZHANG Shusen, et al. Advances and applications in graph neural network[J]. Chinese Journal of Computers, 2022, 45(1): 35-68.
[24]	LIU Ziyu, FENG Ruyi, WANG Lizhe, et al. Dual learning-based graph neural network for remote sensing image super-resolution[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-14.
[25]	白铂, 刘玉婷, 马驰骋, 等. 图神经网络[J]. 中国科学(数学), 2020, 50(3): 367-384.
	BAI Bo, LIU Yuting, MA Chicheng, et al. Graph neural network[J]. Scientia Sinica (Mathematica), 2020, 50(3): 367-384.
[26]	CUI Wei, HE Xin, YAO Meng, et al. Knowledge and spatial pyramid distance-based gated graph attention network for remote sensing semantic segmentation[J]. Remote Sensing, 2021, 13(7): 1312.
[27]	QIN Anyong, SHANG Zhaowei, TIAN Jinyu, et al. Spectral-spatial graph convolutional networks for semisupervised hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(2): 241-245.
[28]	YANG Pan, TONG Lei, QIAN Bin, et al. Hyperspectral image classification with spectral and spatial graph using inductive representation learning network[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 791-800.
[29]	HONG Danfeng, GAO Lianru, YAO Jing, et al. Graph convolutional networks for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(7): 5966-5978.
[30]	刘杰, 尚学群, 宋凌云, 等. 图神经网络在复杂图挖掘上的研究进展[J]. 软件学报, 2022, 33(10): 3582-3618.
	LIU Jie, SHANG Xuequn, SONG Lingyun, et al. Progress of graph neural networks on complex graph mining[J]. Journal of Software, 2022, 33(10): 3582-3618.
[31]	徐冰冰, 岑科廷, 黄俊杰, 等. 图卷积神经网络综述[J]. 计算机学报, 2020, 43(5): 755-780.
	XU Bingbing, CEN Keting, HUANG Junjie, et al. A survey on graph convolutional neural network[J]. Chinese Journal of Computers, 2020, 43(5): 755-780.
[32]	康世泽, 吉立新, 张建朋. 一种基于图注意力网络的异质信息网络表示学习框架[J]. 电子与信息学报, 2021, 43(4): 915-922.
	KANG Shize, JI Lixin, ZHANG Jianpeng. Heterogeneous information network representation learning framework based on graph attention network[J]. Journal of Electronics & Information Technology, 2021, 43(4): 915-922.
[33]	CIANO G, ROSSI A, BIANCHINI M, et al. On inductive-transductive learning with graph neural networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(2): 758-769.
[34]	HAMILTON W, YING Zhitao, LESKOVEC J. Inductive representation learning on large graphs[J]. Advances in Neural Information Processing Systems, 2017, 30: 1024-1034.
[35]	DING Yao, ZHAO Xiaofeng, ZHANG Zhili, et al. Graph sample and aggregate-attention network for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 1-5.
[36]	贾永红, 谢志伟, 吕臻, 等. 一种新的遥感影像变化检测方法[J]. 武汉大学学报(信息科学版), 2016, 41(8): 1001-1006.
	JIA Yonghong, XIE Zhiwei, LÜ Zhen, et al. A new change detection method of remote sensing image[J]. Geomatics and Information Science of Wuhan University, 2016, 41(8): 1001-1006.
[37]	佃袁勇, 方圣辉, 姚崇怀. 多尺度分割的高分辨率遥感影像变化检测[J]. 遥感学报, 2016, 20(1): 129-137.
	DIAN Yuanyong, FANG Shenghui, YAO Chonghuai. Change detection for high-resolution images using multilevel segment method[J]. Journal of Remote Sensing, 2016, 20(1): 129-137.
[38]	SZABÓ S, GÁCSI Z, BALÁZS B. Specific features of NDVI, NDWI and MNDWI as reflected in land cover categories[J]. Landscape & Environment, 2016, 10(3/4): 194-202.
[39]	赵春雷, 钱拴, 黄强, 等. 基于降水量的白洋淀最低水位预测研究[J]. 中国生态农业学报(中英文), 2019, 27(8): 1238-1244.
	ZHAO Chunlei, QIAN Shuan, HUANG Qiang, et al. Prediction of minimum water level in Baiyangdian Lake based on precipitation[J]. Chinese Journal of Eco-Agriculture, 2019, 27(8): 1238-1244.
[40]	季德强, 王海荣, 车淼, 等. KNN-GWD推荐模型及其应用[J]. 应用科学学报, 2022, 40(1): 145-154.
	JI Deqiang, WANG Hairong, CHE Miao, et al. KNN-GWD recommendation model and its application[J]. Journal of Applied Sciences, 2022, 40(1): 145-154.
[41]	ZUO Xibing, YU Xuchu, LIU Bing, et al. Graph inductive learning method for small sample classification of hyperspectral remote sensing images[J]. European Journal of Remote Sensing, 2020, 53(1): 349-357.
[42]	ZHANG Tao, SHAN Haoran, LITTLE M A. Causal GraphSAGE: a robust graph method for classification based on causal sampling[J]. Pattern Recognition, 2022, 128: 108696.
[43]	YANG Kunping, TONG Xinyi, XIA Guisong, et al. Hidden path selection network for semantic segmentation of remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-15.
[44]	DING Lei, TANG Hao, BRUZZONE L. LANet: local attention embedding to improve the semantic segmentation of remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(1): 426-435.
[45]	YU Dawen, JI Shunping. Long-range correlation supervision for land-cover classification from remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 1-14.
[46]	WANG Hong, CHEN Xianzhong, ZHANG Tianxiang, et al. CCTNet: coupled CNN and transformer network for crop segmentation of remote sensing images[J]. Remote Sensing, 2022, 14(9): 1956.

输入：图结构数据G=(V,E)；采样深度K；权重矩阵W^k,∀k∈{1,2,…,K}；平均聚合函数Mean AGG()；采样函数N()
;
for k=1,2,…,K do
	for v∈V do

	end

end

输出：节点向量表示z_v

算法类别	数据集	各类地物分类准确率/（%）				OA/（%）	Kappa系数
算法类别	数据集	建设用地	道路	植被	水体	OA/（%）	Kappa系数
CART	数据A	10.78	31.52	88.36	95.61	52.21	0.39
	数据B	9.71	21.22	86.68	80.44	75.27	0.61
	数据C	13.38	5.23	22.19	70.61	20.77	0.10
GCN	数据A	98.45	7.51	82.73	78.19	84.78	0.76
	数据B	88.42	21.33	98.88	92.64	93.71	0.88
	数据C	97.03	5.15	78.01	5.92	74.48	0.49
GAT	数据A	97.47	7.75	65.15	98.62	86.85	0.79
	数据B	92.62	10.96	98.16	94.26	94.51	0.90
	数据C	96.37	5.38	75.86	5.45	73.62	0.47
LANet	数据A	75.49	33.29	78.78	98.37	80.01	0.71
	数据B	95.76	12.17	98.92	83.01	90.27	0.84
	数据C	96.12	34.93	74.58	51.92	78.80	0.64
CCTNet	数据A	92.40	57.33	77.15	98.96	88.36	0.81
	数据B	90.42	58.98	94.58	85.92	87.48	0.82
	数据C	91.48	50.33	76.28	59.44	81.23	0.67
SLCNet	数据A	96.47	40.31	88.01	99.79	93.87	0.91
	数据B	97.13	55.56	94.77	99.53	96.64	0.94
	数据C	90.71	45.47	81.16	52.62	80.09	0.66
ANF GraphSAGE	数据A	97.32	13.32	97.58	99.22	94.17	0.91
	数据B	97.65	56.40	98.86	97.88	97.50	0.95
	数据C	97.72	39.13	77.55	53.73	83.51	0.69

运行阶段	数据集	CART	GCN	GAT	LANet	CCTNet	SLCNet	ANF GraphSAGE
训练阶段	数据A	13	4439	4483	73 474	84 714	18 035	7352
	数据B	15	7093	7144	73 474	84 714	18 035	10 253
	数据C	12	3829	3873	76 252	85 701	18 154	6850
预测阶段	数据A	3	20	52	23	25	31	22
	数据B	5	22	51	28	22	29	16
	数据C	3	26	52	33	17	34	15

方法	数据集	OA/（%）
G－GraphSAGE	数据A	93.32
	数据B	94.45
	数据C	74.05
S－GraphSAGE	数据A	82.92
	数据B	91.06
	数据C	73.78
MNF GraphSAGE	数据A	94.03
	数据B	97.19
	数据C	83.66
ANF GraphSAGE	数据A	94.17
	数据B	97.50
	数据C	83.51

参数值	k=10	k=15	k=20
λ=0.3	0.88	0.89	0.89
λ=0.6	0.89	0.90	0.88
λ=0.9	0.87	0.86	0.89
自适应权重	0.90	0.91	0.84