Acta Geodaetica et Cartographica Sinica >

2018 , Vol. 47 >Issue 5: 620 - 630

DOI: https://doi.org/10.11947/j.AGCS.2018.20170191

Joint Multi-scale Convolution Neural Network for Scene Classification of High Resolution Remote Sensing Imagery

  • ZHENG Zhuo ,
  • FANG Fang ,
  • LIU Yuanyuan ,
  • GONG Xi ,
  • GUO Mingqiang ,
  • LUO Zhongwen
Expand
  • 1. College of Information Engineering, China University of Geosciences, Wuhan 430074, China;
    2. State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China

Received date: 2017-04-17

  Revised date: 2018-02-09

  Online published: 2018-06-01

Supported by

The National Natural Science Foundation of China (Nos.61602429;41701446),Chinese Geologic Survey Project (No.KZ17Z618)

Fold

Abstract

High resolution remote sensing imagery scene classification is important for automatic complex scene recognition, which is the key technology for military and disaster relief, etc. In this paper, we propose a novel joint multi-scale convolution neural network (JMCNN) method using a limited amount of image data for high resolution remote sensing imagery scene classification. Different from traditional convolutional neural network, the proposed JMCNN is an end-to-end training model with joint enhanced high-level feature representation, which includes multi-channel feature extractor, joint multi-scale feature fusion and Softmax classifier. Multi-channel and scale convolutional extractors are used to extract scene middle features, firstly. Then, in order to achieve enhanced high-level feature representation in a limit dataset, joint multi-scale feature fusion is proposed to combine multi-channel and scale features using two feature fusions. Finally, enhanced high-level feature representation can be used for classification by Softmax. Experiments were conducted using two limit public UCM and SIRI datasets. Compared to state-of-the-art methods, the JMCNN achieved improved performance and great robustness with average accuracies of 89.3% and 88.3% on the two datasets.

Key words： high resolution remote sensing imagery; scene classification; joint multi-scale convolution neural network; enhanced high-level feature representation; limit datasets

Cite this article

ZHENG Zhuo , FANG Fang , LIU Yuanyuan , GONG Xi , GUO Mingqiang , LUO Zhongwen . Joint Multi-scale Convolution Neural Network for Scene Classification of High Resolution Remote Sensing Imagery[J]. Acta Geodaetica et Cartographica Sinica, 2018 , 47(5) : 620 -630 . DOI: 10.11947/j.AGCS.2018.20170191

References

[1] CHERIYADAT A M. Unsupervised Feature Learning for Aerial Scene Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2014, 52(1):439-451.
[2] SERRANO N, SAVAKIS A E, LUO J B. Improved Scene Classification Using Efficient Low-Level Features and Semantic Cues[J]. Pattern Recognition, 2004, 37(9):1773-1784.
[3] 殷慧, 曹永锋, 孙洪. 基于多维金字塔表达和AdaBoost的高分辨率SAR图像城区场景分类算法[J]. 自动化学报, 2010, 36(8):1099-1106. YIN Hui, CAO Yongfeng, SUN Hong. Urban Scene Classification Based on Multi-dimensional Pyramid Representation and AdaBoost Using High Resolution SAR Images[J]. Acta Automatica Sinica, 2010, 36(8):1099-1106.
[4] LAZEBNIK S, SCHMID C, PONCE J. Beyond Bags of Features:Spatial Pyramid Matching for Recognizing Natural Scene Categories[C]//Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York, NY:IEEE, 2006:2169-2178.
[5] YANG Yi, NEWSAM S. Spatial Pyramid Co-occurrence for Image Classification[C]//IEEE International Conference on Computer Vision. Barcelona, Spain:IEEE, 2011:1465-1472.
[6] ZHAO Bei, ZHONG Yanfei, ZHANG Liangpei. Scene Classification via Latent Dirichlet Allocation Using a Hybrid Generative/Discriminative Strategy for High Spatial Resolution Remote Sensing Imagery[J]. Remote Sensing Letters, 2013, 4(12):1204-1213.
[7] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet Classification with Deep Convolutional Neural Networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada:ACM, 2012:1097-1105.
[8] HECHT-NIELSEN R. Theory of the Backpropagation Neural Network[C]//International Joint Conference on Neural Networks. Washington, DC:IEEE, 1989(1):593-605.
[9] 何小飞, 邹峥嵘, 陶超, 等. 联合显著性和多层卷积神经网络的高分影像场景分类[J]. 测绘学报, 2016, 45(9):1073-1080. DOI:10.11947/j.AGCS.2016.20150612. HE Xiaofei, ZOU Zhengrong, TAO Chao, et al. Combined Saliency with Multi-Convolutional Neural Network for High Resolution Remote Sensing Scene Classification[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(9):1073-1080. DOI:10.11947/j.AGCS.2016.20150612.
[10] CASTELLUCCIO M, POGGI G, SANSONE C, et al. Land Use Classification in Remote Sensing Images by Convolutional Neural Networks[J]. Acta Ecologica Sinica, 2015, 28(2):627-635.
[11] PENATTI O A B, NOGUEIRA K, SANTOS J A D. Do Deep Features Generalize from Everyday Objects to Remote Sensing and Aerial Scenes Domains?[C]//IEEE Conference on Computer Vision and Pattern Recognition Workshops. Boston, MA:IEEE, 2015:44-51.
[12] 李学龙, 史建华, 董永生, 等. 场景图像分类技术综述[J]. 中国科学(信息科学), 2015, 45(7):827-848. LI Xuelong, SHI Jianhua, DONG Yongsheng, et al. A Survey on Scene Image Classification[J]. Scientia Sinica (Informationis), 2015, 45(7):827-848.
[13] LI Haoxiang, LIN Zhe, SHEN Xiaohui, et al. A Convolutional Neural Network Cascade for Face Detection[C]//IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA:IEEE, 2015:5325-5334.
[14] GLOROT X, BORDES A, BENGIO Y. Deep Sparse Rectifier Neural Networks[C]//Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics. Fort Lauderdale, Florida:HAL, 2011:315-323.
[15] SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going Deeper with Convolutions[C]//IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA:IEEE, 2015:1-9.
[16] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout:A Simple Way to Prevent Neural Networks from Overfitting[J]. Journal of Machine Learning Research, 2014, 15(1):1929-1958.
[17] ZHAO Bei, ZHONG Yanfei, XIA Guisong, et al. Dirichlet-Derived Multiple Topic Scene Classification Model for High Spatial Resolution Remote Sensing Imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(4):2108-2123.
[18] ZHAO Bei, ZHONG Yanfei, ZHANG Liangpei, et al. The Fisher Kernel Coding Framework for High Spatial Resolution Scene Classification[J]. Remote Sensing, 2016, 8(2):157.
[19] ZHU Qiqi, ZHONG Yanfei, ZHAO Bei, et al. Bag-of-Visual-Words Scene Classifier with Local and Global Features for High Spatial Resolution Remote Sensing Imagery[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(6):747-751.
[20] ABADI M, BARHAM P, CHEN Jianmin, et al. TensorFlow:A System for Large-scale Machine Learning[C]//Proceedings of the 12th Usenix Conference on Operating Systems Design and Implementation. Berkeley, CA:USENIX Association, 2016.
[21] ZHANG Fan, DU Bo, ZHANG Liangpei. Saliency-Guided Unsupervised Feature Learning for Scene Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(4):2175-2184.
[22] LIENOU M, MAITRE H, DATCU M. Semantic Annotation of Satellite Images Using Latent Dirichlet Allocation[J]. IEEE Geoscience and Remote Sensing Letters, 2010, 7(1):28-32.
[23] CHEN Shizhi, TIAN YingLi. Pyramid of Spatial Relations for Scene-level Land Use Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(4):1947-1957.
[24] BREIMAN L. Random Forests[J]. Machine Learning, 2001, 45(1):5-32.
[25] ADANKON M M, CHERIET M. Support Vector Machine[J]. Computer Science, 2002, 1(4):1-28.
Options
Outlines

/

〈 〉