融合半监督学习的无监督遥感影像场景分类

doi:10.11947/j.AGCS.2022.20210270

摘要/Abstract

摘要： 自监督学习可以不依赖样本标签对遥感影像进行特征提取，但是特征分类仍然依赖有监督方法。为了克服有监督特征分类过程的不足，实现遥感影像特征的无监督自动分类，本文提出一种融合半监督学习的无监督语义聚类方法。首先，使用自监督学习提取遥感影像特征，抽象出图像包含的高层语义信息；然后，基于特征相似度寻找每个样本最相似的近邻，使用在线聚类将相似样本聚为一类，训练一个线性分类器；最后，根据聚类结果为高置信度样本生成伪标签，构造标注样本集，使用半监督方法对模型微调。在4个公开遥感影像场景分类数据集EuroSAT、GID、AID和NWPU-RESISC45上进行验证，分类精度分别达到了94.84%、63.55%、76.42%和86.24%。本文方法结合了在线聚类和半监督学习的优点，缓解了已有方法存在的误差积累和样本利用不充分的问题，在完全不使用标注样本的情况下，充分利用自监督特征训练分类模型，对遥感影像进行场景分类，达到接近有监督学习的分类效果，具有良好的应用价值。

关键词: 遥感影像, 场景分类, 自监督, 在线聚类, 半监督

Abstract: Self-supervised learning can extract features from remote sensing images without relying on sample labels, but supervised method is still required to complete feature classification. To overcome the shortcomings of feature classification process and automatically classify remote sensing image features, an unsupervised semantic clustering method based on semi-supervised learning is proposed. First, the features of remote sensing images are extracted using self-supervised learning to abstract the high-level semantic information contained in the images. Then, each sample's closest neighbors are found based on the feature similarity, and a linear classifier is trained by clustering similar samples into one class using the online clustering method. Finally, based on the clustering results, pseudo labels are generated for samples with high confidence to construct a label sample set, and the model is fine-tuned using a semi-supervised method. Four public remote sensing image scene classification datasets, EuroSAT, GID, AID and NWPU-RESISC45, were validated, and the classification accuracy reached 94.84%, 63.55%, 76.42% and 86.24%, respectively. The method presented in this paper combines the advantages of online clustering and semi-supervised learning, alleviates the problems of error accumulation and insufficient sample utilization in existing methods. It makes full use of the self-supervised features to train the classification model and complete the scene classification of remote sensing images without using labeled samples at all. It achieves the classification effect close to supervised learning and has good application value.

Key words: remote sensing, scene classification, self-supervised, online clustering, semi-supervised

中图分类号:

P237

白坤, 慕晓冬, 陈雪冰, 朱永清, 尤轩昂. 融合半监督学习的无监督遥感影像场景分类[J]. 测绘学报, 2022, 51(5): 691-702.

BAI Kun, MU Xiaodong, CHEN Xuebing, ZHU Yongqing, YOU Xuanang. Unsupervised remote sensing image scene classification based on semi-supervised learning[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(5): 691-702.

参考文献 30

[1]	李德仁. 展望5G/6G时代的地球空间信息技术[J]. 测绘学报, 2019, 48(12):1475-1481.DOI:10.11947/j.AGCS.2019.20190437. LI Deren. Towards geospatial information technology in 5G/6G era[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(12):1475-1481 DOI:10.11947/j.AGCS.2019.20190437.
[2]	叶利华,王磊,张文文,等. 高分辨率光学遥感场景分类的深度度量学习方法[J]. 测绘学报, 2019, 48(6):698-707.DOI:10.11947/j.AGCS.2019.20180434. YE Lihua, WANG Lei, ZHANG Wenwen, et al. Deep metric learning method for high resolution remote sensing image scene classification[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6):698-707. DOI:10.11947/j.AGCS.2019.20180434.
[3]	郑卓,方芳,刘袁缘,等. 高分辨率遥感影像场景的多尺度神经网络分类法[J]. 测绘学报, 2018, 47(5):620-630. DOI:10.11947/j.AGCS.2018.20170191. ZHENG Zhuo, FANG Fang, LIU Yuanyuan, et al. Joint multi-scale convolution neural network for scene classification of high resolution remote sensing imagery[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(5):620-630. DOI:10.11947/j.AGCS.2018.20170191.
[4]	ZEILER Matthew D,FERGUS Rob. Visualizing and understanding convolutional networks[C]//Proceedings of 2014 European Conference on Computer Vision.[S.l.]:Springer, 2014:818-833.
[5]	ZHANG R, ISOLA P, EFROS A A. Colorful image colorization[C]//Proceedings of 2016 European Conference on Computer Vision.[S.l.]:Springer, 2016:649-666.
[6]	LARSSON G, MAIRE M, SHAKHNAROVICH G. Colorization as a proxy task for visual understanding[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA:IEEE, 2017:6874-6883.
[7]	KOMODAKIS N,GIDARIS S. Unsupervised representation learning by predicting image rotations[C]//Proceedings of 2018 International Conference on Learning Representations. Vancouver, Canada:ICLR, 2018.
[8]	CHEN Ting,ZHAI Xiaohua,RITTER Marvin, et al. Self-supervised GANs via auxiliary rotation loss[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, Ca, USA:IEEE, 2019:12146-12155.
[9]	KIM D, CHO D, YOO D, et al. Learning image representations by completing damaged jigsaw puzzles[C]//Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision. Lake Tahoe, NV, USA:IEEE, 2018:793-802.
[10]	NOROOZI M, FAVARO P. Unsupervised learning of visual representations by solving jigsaw puzzles[C]//Proceedings of 2016 European Conference on Computer Vision.[S.l.]:Springer, 2016:69-84.
[11]	WEI Chen,XIE Lingxi,REN Xutong, et al. Iterative reorganization with weak spatial constraints:solving arbitrary jigsaw puzzles for unsupervised representation learning[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA:IEEE, 2019:1910-1919.
[12]	CHEN Ting,KORNBLITH Simon,NOROUZI Mohammad, et al. A simple framework for contrastive learning of visual representations[C]//Proceedings of 2020 International Conference on Machine Learning. Addis Ababa, Ethiopia:PMLR, 2020:1597-1607.
[13]	HE Kaiming,FAN Haoqi,WU Yuxin, et al. Momentum contrast for unsupervised visual representation learning[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA:IEEE, 2020:9729-9738.
[14]	TAO Chao, QI Ji, LU Weipeng, et al. Remote sensing image scene classification with self-supervised paradigm under limited labeled samples[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19:1-5.
[15]	CARON M,MISRA I,MAIRAL J, et al. Unsupervised learning of visual features by contrasting cluster assignments[C]//Proceedings of the 34th Conference on Neural Information Processing Systems.[S.l.]:NeurIPS, 2020.
[16]	GRILL J B, STRUB F, ALTCHÉ F, et al. Bootstrap your own latent:a new approach to self-supervised learning[J]. Advances in Neural Information Processing Systems, 2020, 33:21271-21284.
[17]	CHEN Xinlei, HE Kaiming. Exploring simple Siamese representation learning[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, TN, USA:IEEE, 2021:15745-15753.
[18]	ZHAO Zhicheng,LUO Ze,LI Jian, et al. When self-supervised learning meets scene classification:remote sensing scene classification based on a multitask learning framework[J]. Remote Sensing, 2020, 12(20):3276.
[19]	VINCENZI S, PORRELLO A, BUZZEGA P, et al. The color out of space:learning self-supervised representations for Earth Observation imagery[C]//Proceedings of the 25th International Conference on Pattern Recognition (ICPR). Milan, Italy:IEEE, 2021:3034-3041.
[20]	REN Bo, ZHAO Yangyang, HOU Biao, et al. A mutual information-based self-supervised learning model for PolSAR land cover classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(11):9224-9237.
[21]	JUNG H, OH Y, JEONG S, et al. Contrastive self-supervised learning with smoothed representation for remote sensing[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19:1-5.
[22]	CARON M, BOJANOWSKI P, JOULIN A, et al. Deep clustering for unsupervised learning of visual features[C]//Proceedings of 2018 European Conference on Computer Vision.[S.l.]:ECCV,2018.
[23]	VAN GANSBEKE W, VANDENHENDE S, GEORGOULIS S, et al. SCAN:learning to classify images without labels[C]//Proceedings of 2020 Computer Vision.[S.l.]:ECCV,2020.
[24]	MANDAL D, BHARADWAJ S, BISWAS S. A novel self-supervised re-labeling approach for training with noisy labels[C]//Proceedings of 2020 IEEE Winter Conference on Applications of Computer Vision. Snowmass, CO, USA:IEEE, 2020:1370-1379.
[25]	PARK S, HAN S, KIM S, et al. Improving unsupervised image clustering with robust learning[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, TN, USA. IEEE, 2021:12273-12282.
[26]	LI Yunfan, HU Peng, LIU Zitao, et al. Contrastive clustering[C]//Proceedings of 2021 AAAI Conference on Artificial Intelligence. Vancouver, Canada:AAAI, 2021.
[27]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA:IEEE, 2016:770-778.
[28]	BERTHELOT D, CARLINI N, GOODFELLOW I J, et al. MixMatch:a holistic approach to semi-supervised learning[J]. Advances in Neural Information Processing Systems, 2019, 32:1905.
[29]	LU Xiaoqiang,GONG Tengfei,ZHENG Xiangtao. Multisource compensation network for remote sensing cross-domain scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(4):2504-2515.
[30]	WEI Yufan,LUO Xiaobo,HU Lixin, et al. An improved unsupervised representation learning generative adversarial network for remote sensing image scene classification[J]. Remote Sensing Letters, 2020, 11(6):598-607.