Deep metric learning method for high resolution remote sensing image scene classification

doi:10.11947/j.AGCS.2019.20180434

Abstract

Abstract: Due to the similarity of intra-class and dissimilarity of inter-class of high-resolution remote sensing image scene, it is difficult to identify some image scene class. In this paper, a new classification approach for high-resolution remote sensing image scene is proposed based on deep learning and metric learning. Firstly, a clustering center of each class is preset on the output features of deep learning model. Secondly, the Euclidean distance method is used to calculate the average central metric loss. Finally, the final loss function consists of a central metric loss term, a cross entropy loss term, and a weight and bias term. The goal of this method is to improve the classification accuracy by forcing intra-class compactness and inter-class separability. The experimental results show that the proposed method significantly improves the classification accuracy. Compared with state-of-the-art results, the classification accuracy ratios on RSSCN7, UC Merced and NWPU-RESISC45 datasets are increased by 1.46%, 1.09% and 2.51%, respectively.

Key words: deep learning, metric learning, average center metric loss, remote sensing image, scene classification

CLC Number:

P237

YE Lihua, WANG Lei, ZHANG Wenwen, LI Yonggang, WANG Zengkai. Deep metric learning method for high resolution remote sensing image scene classification[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6): 698-707.

References

[1] CHENG Gong, HAN Junwei, LU Xiaoqiang. Remote sensing image scene classification:benchmark and state of the art[J]. Proceedings of the IEEE, 2017, 105(10):1865-1883.
[2] 钱晓亮, 李佳, 程塨, 等. 特征提取策略对高分辨率遥感图像场景分类性能影响的评估[J]. 遥感学报, 2018, 22(5):758-776. QIAN Xiaoliang, LI Jia, CHENG Gong, et al. Evaluation of the effect of feature extraction strategy on the performance of high-resolution remote sensing image scene classification[J]. Journal of Remote Sensing, 2018, 22(5):758-776.
[3] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556, 2014.
[4] SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston:IEEE, 2015:1-9.
[5] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas:IEEE, 2016:770-778.
[6] 郑卓, 方芳, 刘袁缘, 等. 高分辨率遥感影像场景的多尺度神经网络分类法[J]. 测绘学报, 2018, 47(5):620-630. DOI:10.11947/j.AGCS.2018.20170191. ZHENG Zhuo, FANG Fang, LIU Yuanyuan, et al. Joint multi-scale convolution neural network for scene classification of high resolution remote sensing imagery[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(5):620-630. DOI:10.11947/j.AGCS.2018.20170191.
[7] HU Fan, XIA Guisong, HU Jingwen, et al. Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery[J]. Remote Sensing, 2015, 7(11):14680-14707.
[8] CHENG Gong, YANG Ceyuan, YAO Xiwen, et al. When deep learning meets metric learning:remote sensing image scene classification via learning discriminative CNNs[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(5):2811-2821.
[9] LIU Yishu, HUANG Chao. Scene classification via triplet networks[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(1):220-237.
[10] BAO Jiangfeng, CHI Mingmin, BENEDIKTSSON J A. Spectral derivative features for classification of hyperspectral remote sensing images:experimental evaluation[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2013, 6(2):594-601.
[11] JIAO Hongzan, ZHONG Yanfei, ZHANG Liangpei. Artificial DNA computing-based spectral encoding and matching algorithm for hyperspectral remote sensing data[J]. IEEE Transactions on Geoscience and Remote Sensing, 2012, 50(10):4085-4104.
[12] FRANKLIN S E, HALL R J, MOSKAL L M, et al. Incorporating texture into classification of forest species composition from airborne multispectral images[J]. International Journal of Remote Sensing, 2000, 21(1):61-79.
[13] CLAUSI D A, DENG Huang. Design-based texture feature fusion using Gabor filters and co-occurrence probabilities[J]. IEEE Transactions on Image Processing, 2005, 14(7):925-936.
[14] ZHANG Liangpei, HUANG Xin, HUANG Bo, et al. A pixel shape index coupled with spectral information for classification of high spatial resolution remotely sensed imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2006, 44(10):2950-2961.
[15] YANG Yi, NEWSAM S. Bag-of-visual-words and spatial extensions for land-use classification[C]//Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems. New York:ACM, 2010:270-279.
[16] CHEN Shizhi, TIAN Yingli. Pyramid of spatial relatons for scene-level land use classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2014, 53(4):1947-1957.
[17] ZHAO Lijun, TANG Ping, HUO Lianzhi. Land-use scene classification using a concentric circle-structured multiscale bag-of-visual-words model[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 7(12):4620-4631.
[18] LIENOU M, MAITRE H, DATCU M. Semantic annotation of satellite images using latent dirichlet allocation[J]. IEEE Geoscience and Remote Sensing Letters, 2010, 7(1):28-32.
[19] ZHONG Yanfei, ZHU Qiqi, ZHANG Liangpei. Scene classification based on the multifeature fusion probabilistic topic model for high spatial resolution remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(11):6207-6222.
[20] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553):436-444.
[21] LUUS F P S, SALMON B P, VAN DEN BERGH F, et al. Multiview deep learning for land-use classification[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(12):2448-2452.
[22] ZHANG Fan, DU Bo, ZHANG Liangpei. Scene classification via a gradient boosting random convolutional network framework[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(3):1793-1802.
[23] WU Hang, LIU Baozhen, SU Weihua, et al. Deep filter banks for land-use scene classification[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(12):1895-1899.
[24] ZENG Dan, CHEN Shuaijun, CHEN Boyang, et al. Improving remote sensing scene classification by integrating global-context and local-object features[J]. Remote Sensing, 2018, 10(5):734.
[25] XIA Guisong, HU Jingwen, HU Fan, et al. AID:A benchmark data set for performance evaluation of aerial scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7):3965-3981.
[26] YE Lihua, WANG Lei, SUN Yaxin, et al. Aerial scene classification via an ensemble extreme learning machine classifier based on discriminative hybrid convolutional neural networks features[J]. International Journal of Remote Sensing, 2018, 40(7):2759-2783.
[27] 许夙晖, 慕晓冬, 赵鹏, 等. 利用多尺度特征与深度网络对遥感影像进行场景分类[J]. 测绘学报, 2016, 45(7):834-840. DOI:10.11947/j.AGCS.2016.20150623. XU Suhui, MU Xiaodong, ZHAO Peng, et al. Scene classification of remote sensing image based on multi-scale feature and deep neural network[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(7):834-840. DOI:10.11947/j.AGCS.2016.20150623.
[28] YE Lihua, WANG Lei, SUN Yaxin, et al. Parallel multi-stage features fusion of deep convolutional neural networks for aerial scene classification[J]. Remote Sensing Letters, 2018, 9(3):294-303.
[29] WEN Yandong, ZHANG Kaipeng, LI Zhifeng, et al. A discriminative feature learning approach for deep face recognition[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands:Springer, 2016:499-515.
[30] BIAN Xiaoyong, CHEN Chen, TIAN Long, et al. Fusing local and global features for high-resolution scene classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(6):2889-2901.