一种联合空谱特征的高光谱影像分类胶囊网络

doi:10.11947/j.AGCS.2023.20220565

摘要/Abstract

摘要： 高效稳定的深度学习分类器有助于提升高光谱遥感影像的分类精度。针对卷积神经网络标量式神经元特征表达能力有限、无法有效建模特征之间空间层次结构关系的不足，设计了一种考虑数据图谱合一特性的端到端高光谱胶囊网络（H-CapsNet）。H-CapsNet主体由编码器（卷积层、PrimaryCaps层及DigitCats层）和解码器（全连接层）组成，通过在网络输入端嵌入通道和空间注意力模块，以此增强模型对空谱特征的抓取和识别，进而提升网络对特征的聚焦和表达能力。以资源一号02D卫星获取的张家港高光谱影像及公共数据集University of Pavia和University of Houston影像为例进行试验，将H-CapsNet网络与传统机器学习算法和多个深度学习网络进行对比。试验结果表明，在3景不同分辨率的高光谱影像上，H-CapsNet分类网络均取得了最优的分类效果，总体精度相较于其他方法分别提升了2.36%~7.67%、0.16%~11.8%和1.75%~15.58%。H-CapsNet网络对小像素邻域具有较好的适应性，当图像块尺寸有限时，仍可以取得相对理想的分类结果。

关键词: 胶囊网络, 深度学习, 高光谱遥感, 资源一号02D, 土地覆盖分类

Abstract: An efficient and stable deep learning classifier can improve the classification accuracy of hyperspectral remote sensing images. In order to deal with the insufficiency of the scalar neuron's limited feature expression ability and the inability to effectively model the spatial hierarchical relationship among features in convolutional neural networks, an end-to-end hyperspectral capsule network (H-CapsNet) was designed considering the characteristics of hyperspectral image. The main body of H-CapsNet is composed of encoder (Conv, PrimaryCaps and DigitCats) and decoder (fully connection layer). It mainly embeds channel and spatial attention modules at the network input to enhance the model's capture and recognition of spatial and spectral features, thereby improving the network's ability to focus and express features. Taking the hyperspectral images of Zhang-jia-gang city and two public datasets:University of Pavia and University of Houston, as examples, the performance of the proposed H-CapsNet was compared with traditional machine learning algorithms and several deep neural networks. The experimental results show that the H-CapsNet has achieved the best classification accuracy on three hyperspectral images with different resolutions, and the overall accuracy is improved by 2.36%~7.67%, 0.16%~11.8% and 1.75%~15.58% compared with other methods. In particular, the H-CapsNet has good adaptability to small pixel neighborhoods. When the image patch size is limited, it can still achieve relatively ideal classification results.

Key words: capsule network, deep learning, hyperspectral remote sensing, ZY-1 02D, land cover classification

中图分类号:

TP751:P237

杜培军, 张伟, 张鹏, 林聪, 郭山川, 胡泽周. 一种联合空谱特征的高光谱影像分类胶囊网络[J]. 测绘学报, 2023, 52(7): 1090-1104.

DU Peijun, ZHANG Wei, ZHANG Peng, LIN Cong, GUO Shanchuan, HU Zezhou. A capsule network for hyperspectral image classification employing spatial-spectral feature[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(7): 1090-1104.

参考文献

[1] 童庆禧, 张兵, 张立福. 中国高光谱遥感的前沿进展[J]. 遥感学报, 2016, 20(5):689-707. TONG Qingxi, ZHANG Bing, ZHANG Lifu. Current progress of hyperspectral remote sensing in China[J]. Journal of Remote Sensing, 2016, 20(5):689-707.
[2] GOETZ A F H. Three decades of hyperspectral remote sensing of the Earth:a personal view[J]. Remote Sensing of Environment, 2009, 113:S5-S16.
[3] 张良培, 李家艺. 高光谱图像稀疏信息处理综述与展望[J]. 遥感学报, 2016, 20(5):1091-1101. ZHANG Liangpei, LI Jiayi. Development and prospect of sparse representation-based hyperspectral image processing and analysis[J]. Journal of Remote Sensing, 2016, 20(5):1091-1101.
[4] 杜培军, 夏俊士, 薛朝辉, 等. 高光谱遥感影像分类研究进展[J]. 遥感学报, 2016, 20(2):236-256. DU Peijun, XIA Junshi, XUE Zhaohui, et al. Review of hyperspectral remote sensing image classification[J]. Journal of Remote Sensing, 2016, 20(2):236-256.
[5] GHAMISI P, MAGGIORI E, LI S T, et al. New frontiers in spectral-spatial hyperspectral image classification the latest advances based on mathematical morphology, Markov random fields, segmentation, sparse representation, and deep learning[J]. IEEE Geoscience and Remote Sensing Magazine, 2018, 6(3):10-43.
[6] ZHONG Y, HU X, LUO C, et al. WHU-Hi:UAV-borne hyperspectral with high spatial resolution (H²) benchmark datasets and classifier for precise crop identification based on deep convolutional neural network with CRF[J]. Remote Sensing of Environment, 2020, 250:112012.
[7] TAN K, WANG H, CHEN L, et al. Estimation of the spatial distribution of heavy metal in agricultural soils using airborne hyperspectral imaging and random forest[J]. Journal of Hazardous Materials, 2020, 382:120987.
[8] DARVISHZADEH R, ATZBERGER C, SKIDMORE A, et al. Mapping grassland leaf area index with airborne hyperspectral imagery:a comparison study of statistical approaches and inversion of radiative transfer models[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2011, 66(6):894-906.
[9] NASRABADI N M. Hyperspectral target detection:an overview of current and future challenges[J]. IEEE Signal Processing Magazine, 2014, 31(1):34-44.
[10] XUE Z, NIE X. Low-rank and sparse representation with adaptive neighborhood regularization for hyperspectral image classification[J]. Journal of Geodesy and Geoinformation Science, 2022, 5(1):73-90.
[11] FU P, ZHANG W, YANG K, et al. A novel spectral analysis method for distinguishing heavy metal stress of maize due to copper and lead:RDA and EMD-PSD[J]. Ecotoxicology and Environmental Safety, 2020, 206:111211.
[12] LIU S, MARINELLI D, BRUZZONE L, et al. A review of change detection in multitemporal hyperspectral images:current techniques, applications, and challenges[J]. IEEE Geoscience and Remote Sensing Magazine, 2019, 7(2):140-58.
[13] CHANG S, DU B, ZHANG L. A subspace selection-based discriminative forest method for hyperspectral anomaly detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(6):4033-4046.
[14] JIA S, TANG G, ZHU J, et al. A novel ranking-based clustering approach for hyperspectral band selection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(1):88-102.
[15] TARAMELLI A, VALENTINI E, INNOCENTI C, et al. FHYL:field spectral libraries, airborne hyperspectral images and topographic and bathymetric LiDAR data for complex coastal mapping[C]//Proceedings of 2013 IEEE International Geoscience and Remote Sensing Symposium. Melbourne:IEEE, 2013.
[16] SU H, YAO W, WU Z, et al. Kernel low-rank representation with elastic net for China coastal wetland land cover classification using GF-5 hyperspectral imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 171:238-252.
[17] LI Z, HUANG H, DUAN Y, et al. DLPNet:a deep manifold network for feature extraction of hyperspectral imagery[J]. Neural Networks, 2020, 129:7-18.
[18] KUMAR B, DIKSHIT O, GUPTA A, et al. Feature extraction for hyperspectral image classification:a review[J]. International Journal of Remote Sensing, 2020, 41(16):6248-6287.
[19] GHAMISI P, PLAZA J, CHEN Y, et al. Advanced spectral classifiers for hyperspectral images:a review[J]. IEEE Geoscience and Remote Sensing Magazine, 2017, 5(1):8-32.
[20] DU P J, XIA J S, ZHANG W, et al. Multiple classifier system for remote sensing image classification:a review[J]. Sensors (Basel), 2012, 12(4):4764-4792.
[21] BENEDIKTSSON J A, PESARESI M, ARNASON K. Classification and feature extraction for remote sensing images from urban areas based on morphological transformations[J]. IEEE Transactions on Geoscience and Remote Sensing, 2003, 41(9):1940-1949.
[22] DALLA M M, VILLA A, BENEDIKTSSON J A, et al. Classification of hyperspectral images by using extended morphological attribute profiles and independent component analysis[J]. IEEE Geoscience and Remote Sensing Letters, 2011, 8(3):542-546.
[23] JIA S, SHEN L L, LI Q Q. Gabor feature-based collaborative representation for hyperspectral imagery classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(2):1118-1129.
[24] ZHANG W, DU P J, LIN C, et al. An improved feature set for hyperspectral image classification:harmonic analysis optimized by multiscale guided filter[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13:3903-3916.
[25] XIA J S, DU P J, HE X Y, et al. Hyperspectral remote sensing image classification based on rotation forest[J]. IEEE Geoscience and Remote Sensing Letters, 2014, 11(1):239-243.
[26] 魏立飞, 余铭, 钟燕飞, 等. 空-谱融合的条件随机场高光谱影像分类方法[J]. 测绘学报, 2020, 49(03):343-354. DOI:10.11947/j.AGCS.2020.20190042. WEI Lifei, YU Ming, ZHONG Yanfei, et al. Hyperspectral image classification method based on space-spectral fusion conditional random field[J].Acta Geodaetica et Cartographica Sinica, 2020, 49(3):343-354. DOI:10.11947/j.AGCS.2020.20190042.
[27] YUAN Q, SHEN H, LI T, et al. Deep learning in environmental remote sensing:achievements and challenges[J]. Remote Sensing of Environment, 2020, 241:111716.
[28] MA L, LIU Y, ZHANG X, et al. Deep learning in remote sensing applications:a meta-analysis and review[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 152:166-177.
[29] AUDEBERT N, LE SAUX B, LEFEVRE S. Deep learning for classification of hyperspectral data:a comparative review[J]. IEEE Geoscience and Remote Sensing Magazine, 2019, 7(2):159-173.
[30] HONG D, GAO L, YAO J, et al. Graph convolutional networks for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(7):5966-5978.
[31] XUE Z, ZHANG M, LIU Y, et al. Attention-based second-order pooling network for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021:1-16.
[32] HINTON G E, KRIZHEVSKY A, WANG S D. Transforming auto-encoders[C]//Proceedings of the 21st international conference on artificial neural networks. Espoo:Springer Berlin Heidelberg, 2011:44-51.
[33] SABOUR S, FROSST N, HINTON G E. Dynamic routing between capsules[J]. Advances in Neural Information Processing Systems, 2017, 30.
[34] HINTON G E, SABOUR S, FROSST N. Matrix capsules with EM routing[C]//Proceedings of 2018 International Conference on Learning Representations. Vancouver:OpenReview.net, 2018:1-15.
[35] PAOLETTI M E, HAUT J M, FERNANDEZ B, et al. Capsule networks for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(4):2145-2160.
[36] WANG X, TAN K, DU Q, et al. Caps-TripleGAN:GAN-assisted CapsNet for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(9):7232-7245.
[37] JIANG X, LIU W, ZHANG Y, et al. Spectral-spatial hyperspectral image classification using dual-channel capsule networks[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(6):1094-1098.
[38] YIN J, LI S, ZHU H, et al. Hyperspectral image classification using CapsNet with well-initialized shallow layers[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(7):1095-1099.
[39] LI H C, WANG W Y, PAN L, et al. Robust capsule network based on maximum correntropy criterion for hyperspectral image classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13:738-751.
[40] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553):436-444.
[41] GU J, WANG Z, KUEN J, et al. Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77:354-377.
[42] WOO S, PARK J, LEE J-Y, et al. CBAM:convolutional block attention module[C]//Proceedings of 2018 European Conference on Computer Vision. Munich:Springer, 2018:3-19.
[43] WANG Q, WU B, ZHU P, et al. ECA-Net:efficient channel attention for deep convolutional neural networks[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle:IEEE, 2020:11534-11542.
[44] DEBES C, MERENTITIS A, HEREMANS R, et al. Hyperspectral and LiDAR data fusion:outcome of the 2013 GRSS data fusion contest[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6):2405-2418.
[45] 高奎亮, 余旭初, 张鹏强, 等. 利用胶囊网络实现高光谱影像空谱联合分类[J]. 武汉大学学报(信息科学版), 2022, 47(03):428-437. GAO Kuiliang, YU Xuchu, ZHANG Pengqiang, et al. Hyperspectral image spatial-spectral classification using a capsule network based method[J]. Geomatics and Information Science of Wuhan University, 2022, 47(3):428-437.
[46] MOUNTRAKIS G, IM J, OGOLE C. Support vector machines in remote sensing:a review[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2011, 66(3):247-259.
[47] BELGIU M, DRǍGUŢ L. Random forest in remote sensing:a review of applications and future directions[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 114:24-31.
[48] XU Y, ZHANG L, DU B, et al. Spectral-spatial unified networks for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018:1-17.
[49] HU W, HUANG Y, WEI L, et al. Deep convolutional neural networks for hyperspectral image classification[J]. Journal of Sensors, 2015:1-12.
[50] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words:transformers for image recognition at scale[C]//Proceedings of 2021 International Conference on Learning Representations. Vienna:OpenReview.net, 2021:1-22.
[51] SHARMA V, DIBA A, TUYTELAARS T, et al. Hyperspectral CNN for image classification & band selection, with application to face recognition[R]. Belgium:KU Leuven, 2016.
[52] CHEN Y, JIANG H, LI C, et al. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(10):6232-6251.