High spatial resolution imagery scene classification based on semi-supervised CNNs

doi:10.11947/j.AGCS.2021.20200017

Abstract

Abstract: The large amount of labeled dataset is always required to train the deep convolutional neural networks (CNNs) for high spatial resolution (HSR) imagery scene classification. However, the generalization of the learned deep features is decayed when limited labeled data is available. To solve this problem, the scene classification based on semi-supervised CNNs (3sCNN) is proposed. In the proposed method, the labeled data is first used to train the model and then the prediction label and confidence of the unlabeled data is obtained with the trained model. Finally, the unlabeled data with high confidence is considered as the labeled data to train the network again and the progress described above is repeated. To demonstrate the effectiveness of the proposed method, the experiments on three datasets are performed. The results show that the proposed method can effectively improve the classification.

Key words: convolutional neural networks, high spatial resolution imagery, semi-supervised, classification

CLC Number:

P227

YANG Qiulian, LIU Yanfei, DING Lele, MENG Fanxiao. High spatial resolution imagery scene classification based on semi-supervised CNNs[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(7): 930-938.

References

[1] 杜培军. 高分辨率卫星遥感的发展及其在矿山的应用[J]. 2001, 10(1): 5-7. DU Peijun. Development of high-resolution satellite remote sensing and its application in mines[J]. Coal, 2001, 10(1): 5-7.
[2] 曹福成. 高分系列遥感卫星布设中国太空“慧眼”: 我国高分专项建设回眸[J]. 中国军转民, 2015(1): 28-33. CAO Fucheng. Gaofen series remote sensing satellite China’s space “eye”:review of China’s special construction of high marks [J]. Defense Industry Conversion in China, 2015(1): 28-33.
[3] 明冬萍, 骆剑承, 沈占锋, 等. 高分辨率遥感影像信息提取与目标识别技术研究[J]. 测绘科学, 2005, 30(3): 18-20, 3. MING Dongping, LUO Jiancheng, SHEN Zhanfeng, et al. Research on information extraction and target recognition from high resolution remote sensing image[J]. Science of Surveying and Mapping, 2005, 30(3): 18-20, 3.
[4] 周成虎, 骆剑承. 高分辨率卫星遥感影像地学计算[M]. 北京: 科学出版社, 2009. ZHOU Chenghu, LUO Jiancheng.Geologic calculation of high resolution satellite remote sensing image[M]. Beijing: Science Press, 2009.
[5] 张乐飞. 遥感影像的张量表达与流形学习方法研究[D]. 武汉大学, 2013. ZHANG Lefei.Tensor representation and manifold learning methods for remote sensing images[D].Wuhan: Wuhan University, 2013.
[6] 赵济. 面向高分辨率遥感影像分类的条件随机场模型研究[D]. 武汉: 武汉大学, 2017. ZHAO Ji. Conditional random fields for high resolution remote sensing image classification[D]. Wuhan: Wuhan University, 2017.
[7] 孙显, 付琨, 王宏琦. 高分辨率遥感图像理解[M]. 北京: 科学出版社, 2011. SUN Xian, FU Kun, WANG Hongqi. High resolution remote sensing image understanding[M]. Beijing: Science Press, 2011.
[8] 张帆. 面向高分辨率遥感影像分析的深度学习方法研究[D]. 武汉: 武汉大学, 2017. ZHANG Fan. Deep learning for very high resolution remote sensing data analysis[D]. Wuhan: Wuhan University, 2017.
[10] 朱祺琪. 面向高分辨率遥感影像场景语义理解的概率主题模型研究[D]. 武汉: 武汉大学, 2018. ZHU Qiqi. Topic model for high resolution remote sensing imagery semantic scene understanding[D]. Wuhan: Wuhan University, 2018.
[11] ZHONG Y, WU S, ZHAO B. Scene semantic understanding based on the spatial context relations of multiple objects[J]. Remote Sensing, 2017, 9(10):1030.
[12] ZHANG Xiuyuan, DU Shihong, ZHANG Yuan. Semantic and spatial co-occurrence analysis on object pairs for urban scene classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(8): 2630-2643.
[13] ZHU Qiqi, ZHONG Yanfei, ZHANG Liangpei, et al. Scene classification based on the fully sparse semantic topic model[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(10): 5525-5538.
[14] ZHAO Bei, ZHONG Yanfei, XIA Guisong, et al. Dirichlet-derived multiple topic scene classification model for high spatial resolution remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(4): 2108-2123.
[15] XIA Guisong, HU Jingwen, HU Fan, et al. AID: a benchmark data set for performance evaluation of aerial scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7): 3965-3981.
[16] GONG Zhiqiang, ZHONG Ping, YU Yang, et al. Diversity-promoting deep structural metric learning for remote sensing scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(1): 371-390.
[17] ZHANG Wei, TANG Ping, ZHAO Lijun. Remote sensing image scene classification using CNN-CapsNet[J]. Remote Sensing, 2019, 11(5): 494.
[18] LIU Qingshan, HANG Renlong, SONG Huihui, et al. Learning multiscale deep features for high-resolution satellite image scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(1): 117-126.
[19] CHENG Gong, HAN Junwei, LU Xiaoqiang. Remote sensing image scene classification: benchmark and state of the art[J]. Proceedings of the IEEE, 2017, 105(10): 1865-1883.
[20] NOGUEIRA K, PENATTI O A B, DOS SANTOS J A. Towards better exploiting convolutional neural networks for remote sensing scene classification[J]. Pattern Recognition, 2017, 61: 539-556.
[21] ZHAO Lijun, TANG Ping, HUO Lianzhi. A 2-D wavelet decomposition-based bag-of-visual-words model for land-use scene classification[J]. International Journal of Remote Sensing, 2014, 35(6): 2296-2310.
[22] CHEN Shizhi, TIAN Yingli. Pyramid of spatial relations for scene-level land use classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(4): 1947-1957.
[23] ZHANG Fan, DU Bo, ZHANG Liangpei. Saliency-guided unsupervised feature learning for scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(4): 2175-2184.
[24] HAN Xiaobing, ZHONG Yanfei, ZHAO Bei, et al. Scene classification based on a hierarchical convolutional sparse auto-encoder for high spatial resolution imagery[J]. International Journal of Remote Sensing, 2017, 38(2): 514-536.
[25] ZHONG Yanfei, FEI Feng, LIU Yanfei, et al. SatCNN: satellite image dataset classification using agile convolutional neural networks[J]. Remote Sensing Letters, 2017, 8(2): 136-145.
[26] ZHANG Fan, DU Bo, ZHANG Liangpei. Scene classification via a gradient boosting random convolutional network framework[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(3): 1793-1802.
[27] WANG Qi, LIU Shaoteng, CHANUSSOT J, et al. Scene classification with recurrent attention of VHR remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(2): 1155-1167.
[28] CHENG Gong, YANG Ceyuan, YAO Xiwen, et al. When deep learning meets metric learning: remote sensing image scene classification via learning discriminative CNNs[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(5): 2811-2821.
[29] ZHONG Yanfei, FEI Feng, ZHANG Liangpei. Large patch convolutional neural networks for the scene classification of high spatial resolution imagery[J]. Journal of Applied Remote Sensing, 2016, 10(2):025006.
[30] GONG Zhiqiang, ZHONG Ping, HU Weidong, et al. An end-to-end joint unsupervised learning of deep model and pseudo-classes for remote sensing scene representation[C]//Proceedings of 2019 International Joint Conference on Neural Networks (IJCNN). Budapest, Hungary. IEEE, 2019: 1-7.
[31] ZHANG Jun, LIU Jiao, PAN Bin, et al. Domain adaptation based on correlation subspace dynamic distribution alignment for remote sensing image scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(11): 7920-7930.
[32] SONG Shaoyue, YU Hongkai, MIAO Zhenjiang, et al. Domain adaptation for convolutional neural networks-based remote sensing scene classification[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(8): 1324-1328.
[33] OTHMAN E, BAZI Y, MELGANI F, et al. Domain adaptation network for cross-scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(8): 4441-4456.
[34] ZHANG Jun, ZHANG Min, PAN Bin, et al. Semisupervised center loss for remote sensing image scene classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 1362-1373.
[35] YAN Peiyao, HE Feng, YANG Yajie, et al. Semi-supervised representation learning for remote sensing image classification based on generative adversarial networks[J]. IEEE Access, 2020, 8: 54135-54144.
[36] DAI Xueyuan, WU Xiaofeng, WANG Bin, et al. Semisupervised scene classification for remote sensing images: a method based on convolutional neural networks and ensemble learning[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(6): 869-873.
[37] HAN Wei, FENG Ruyi, WANG Lizhe, et al. A semi-supervised generative framework with deep learning features for high-resolution remote sensing image scene classification[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 145: 23-43.
[38] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA. IEEE, 2016: 770-778.
[39] YANG Yi, NEWSAM S. Bag-of-visual-words and spatial extensions for land-use classification[C]//Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems. New York: ACM Press, 2010: 270-279.

[1]	BAI Kun, MU Xiaodong, CHEN Xuebing, ZHU Yongqing, YOU Xuanang. Unsupervised remote sensing image scene classification based on semi-supervised learning [J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(5): 691-702.
[2]	WANG Dandi, XING Shuai, XU Qing, LIN Yuzhun, LI Pengcheng. Automatic sea-land waveform classification method for single-wavelength airborne LiDAR bathymetry [J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(5): 750-761.
[3]	LI Fan, JIN Shaohua, BIAN Gang, CUI Yang, TANG Yulin, ZHANG Yonghou. Construction and verification of improved error model of MBES [J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(5): 762-771.
[4]	ZHANG Yuxin, YAN Qingsong, DENG Fei. Multi-path RSU network method for high-resolution remote sensing image building extraction [J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(1): 135-144.
[5]	FANG Lina, WANG Shuang, ZHAO Zhiyuan, FU Huasheng, CHEN Chongcheng. Automatic classification and vectorization of road markings from mobile laser point clouds [J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(9): 1251-1265.
[6]	JI Xue, TANG Qiuhua, CHEN Yilan, LI Jie, DING Deqiu. Multibeam acoustic seabed classification combining SVM and adaptive boosting algorithm [J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(7): 972-981.
[7]	WANG Si, WANG Guangxia, TIAN Jiangpeng. Classification model of ubiquitous map information facing location-based aggregation [J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(6): 789-799.
[8]	YE Famao, MENG Xianglong, DONG Meng, Nie Yunju, GE Yun, CHEN Xiaoyong. Remote sensing image retrieval with ant colony optimization and a weighted image-to-class distance [J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(5): 612-620.
[9]	SHI Huihui, XU Yannan, TENG Wenxiu, WANG Ni. Scene classification of high-resolution remote sensing imagery based on deep transfer deformable convolutional neural networks [J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(5): 652-663.
[10]	ZHANG Hong, WU Zhiwei, WANG Jicheng, GAO Peichao. Unsupervised band selection for hyperspectral image classification using the Wasserstein metric-based configuration entropy [J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(3): 405-415.
[11]	WANG Chang, ZHANG Yongsheng, JI Song, ZHANG Lei. Multi-feature fusion and random multi-graph synthetic building change method [J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(2): 235-247.
[12]	FANG Lina, SHEN Guixi, YOU Zhilong, GUO Yingya, FU Huasheng, ZHAO Zhiyuan, CHEN Chongcheng. A joint network of point cloud and multiple views for roadside objects recognition from mobile laser point clouds [J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(11): 1558-1573.
[13]	FU Chen, HUANG Shengke, TANG Yan, WU Hangbin, LIU Chun, YAO Lianbi, HUANG Wei. A real-time map matching method for road network using driving scenario classification [J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(11): 1617-1627.
[14]	ZUO Xibing, LIU Bing, YU Xuchu, ZHANG Pengqiang, GAO Kuiliang, ZHU Enze. Graph convolutional network method for small sample classification of hyperspectral images [J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(10): 1358-1369.
[15]	YANG Fanlin, ZHU Zhengren, LI Jiabiao, FENG Chengkai, XING Zhe, WU Ziyin. Seafloor classification based on combined multibeam bathymetry and backscatter using deep convolution neural network [J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(1): 71-84.