Deep 3D convolutional network combined with spatial-spectral features for hyperspectral image classification

doi:10.11947/j.AGCS.2019.20170578

Abstract

Abstract:

A classification method of hyperspectral images based on deep 3D convolution networks is proposed in order to deal with the high dimensional and small samples of hyperspectral image classification. The method first uses hyperspectral data cube as input, and uses 3D convolution operation to extract 3D spatial-spectral features of hyperspectral data cube. Then, the residual learning is used to construct the deep network and extract higher level feature expression to improve the classification accuracy. Finally, the Dropout regularization method is used to prevent overfitting. Experiments were conducted on the University of Pavia, Indian Pines and Salinas datasets, and the results demonstrate that compared with support vector machine and the existing deep learning classification method for hyperspectral images, the method can effectively improve the classification accuracy of hyperspectral image.

Key words: hyperspectral image classification, convolutional neural network, 3D convolution, residual learning

CLC Number:

P237

LIU Bing, YU Xuchu, ZHANG Pengqiang, TAN Xiong. Deep 3D convolutional network combined with spatial-spectral features for hyperspectral image classification[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1): 53-63.

References

[1] CAMPS-VALLS G, BRUZZONE L. Kernel-based methods for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2005, 43(6):1351-1362.
[2] 唐中奇, 付光远, 陈进, 等. 基于多尺度分割的高光谱图像稀疏表示与分类[J]. 光学精密工程, 2015, 23(9):2708-2714. TANG Zhongqi, FU Guangyuan, CHEN Jin, et al. Multiscale segmentation-based sparse coding for hyperspectral image classification[J]. Optics and Precision Engineering, 2015, 23(9):2708-2714.
[3] DÓPIDO I, LI Jun, MARPU P R, et al. Semisupervised self-learning for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013, 51(7):4032-4044.
[4] 田彦平, 陶超, 邹峥嵘, 等. 主动学习与图的半监督相结合的高光谱影像分类[J]. 测绘学报, 2015, 44(8):919-926. DOI:10.11947/j.AGCS.2015.20140221. TIAN Yanping, TAO Chao, ZOU Zhengrong, et al. Semi-supervised graph-based hyperspectral image classification with active learning[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(8):919-926. DOI:10.11947/j.AGCS.2015.20140221.
[5] CHEN Yushi, LIN Zhouhan, ZHAO Xing, et al. Deep learning-based classification of hyperspectral data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6):2094-2107.
[6] HU Wei, HUANG Yangyu, LI Wei, et al. Deep convolutional neural networks for hyperspectral image classification[J]. Journal of Sensors, 2015, 2015(7):220-231.
[7] CHEN Yushi, ZHAO Xing, JIA Xiuping. Spectral-spatial classification of hyperspectral data based on deep belief network[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2015, 8(6):2381-2392.
[8] MOU Lichao, GHAMISI P, ZHU Xiaoxiang. Deep recurrent neural networks for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7):3639-3655.
[9] YUE Jun, ZHAO Wenzhi, MAO Shanjun, et al. Spectral-spatial classification of hyperspectral images using deep convolutional neural networks[J]. Remote Sensing Letters, 2015, 6(6):468-477.
[10] YUE Jun, MAO Shanjun, LI Mei. A deep learning framework for hyperspectral image classification using spatial pyramid pooling[J]. Remote Sensing Letters, 2016, 7(9):875-884.
[11] GHAMISI P, CHEN Yushi, ZHU Xiaoxiang. A self-improving convolution neural network for the classification of hyperspectral data[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(10):1537-1541.
[12] MEI Shaohui, JI Jingyu, HOU Junhui, et al. Learning sensor-specific spatial-spectral features of hyperspectral images via convolutional neural networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(8):4520-4533.
[13] LIU Bing, YU Xuchu, ZHANG Pengqiang, et al. Supervised deep feature extraction for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(4):1909-1921.
[14] LIU Peng, ZHANG Hui, EOM K B. Active deep learning for classification of hyperspectral images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(2):712-724.
[15] LIU Bing, YU Xuchu, ZHANG Pengqiang, et al. A semi-supervised convolutional neural network for hyperspectral image classification[J]. Remote Sensing Letters, 2017, 8(9):839-848.
[16] JIAO Licheng, LIANG Miaomiao, CHEN Huan, et al. Deep fully convolutional network-based spatial distribution prediction for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(10):5585-5599.
[17] JI Shuiwang, XU Wei, YANG Ming, et al. 3D convolutional neural networks for human action recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1):221-231.
[18] 尹宝才, 王文通, 王立春. 深度学习研究综述[J]. 北京工业大学学报, 2015, 41(1):48-59. YIN Baocai, WANG Wentong, WANG Lichun. Review of deep learning[J]. Journal of Beijing University of Technology, 2015, 41(1):48-59.
[19] HE Kaiming, ZHANG Xianyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas:IEEE, 2016:770-778.
[20] NAIR V, HINTON G E. Rectified linear units improve restricted Boltzmann machines[C]//Proceedings of the 27th International Conference on International Conference on Machine Learning. Haifa:Omnipress, 2010:807-814.
[21] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe:Curran Associates Inc, 2012:1097-1105.
[22] 黄鸿, 郑新磊. 高光谱影像空-谱协同嵌入的地物分类算法[J]. 测绘学报, 2016, 45(8):964-972. DOI:10.11947/j.AGCS.2016.20150654. HUANG Hong, ZHENG Xinlei. Hyperspectral image land cover classification algorithm based on spatial-spectral coordination embedding[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(8):964-972. DOI:10.11947/j.AGCS.2016.20150654.
[23] 张春森, 郑艺惟, 黄小兵, 等. 高光谱影像光谱-空间多特征加权概率融合分类[J]. 测绘学报, 2015, 44(8):909-918. DOI:10.11947/j.AGCS.2015.20140544. ZHANG Chunsen, ZHENG Yiwei, HUANG Xiaobing, et al. Hyperspectral image classification based on the weighted probabilistic fusion of multiple spectral-spatial features[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(8):909-918. DOI:10.11947/j.AGCS.2015.20140544.
[24] LI Wei, WU Guodong, ZHANG Fan, et al. Hyperspectral image classification using deep pixel-pair features[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(2):844-853.
[25] BENEDIKTSSON J A, PALMASON J A, SVEINSSON J R. Classification of hyperspectral data from urban areas based on extended morphological profiles[J]. IEEE Transactions on Geoscience and Remote Sensing, 2005, 43(3):480-491. DOI:10.1109/TGRS.2004.842478.
[26] KINGMA D P, BA J. ADAM:A method for Stochastic optimization[R]. The 3rd International Conference for Learning Representations. San Diego:[s.n.], 2015.
[27] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout:a simple way to prevent neural networks from overfitting[J]. The Journal of Machine Learning Research, 2014, 15(1):1929-1958.