测绘学报 ›› 2025, Vol. 54 ›› Issue (12): 2233-2246.doi: 10.11947/j.AGCS.2025.20250293

• 摄影测量学与遥感 • 上一篇    下一篇

基于CNN-ViT混合特征优化的小样本高光谱图像分类

张津1(), 冯凡1(), 戴晨光1, 张振超1, 于英1, 刘冰2   

  1. 1.信息工程大学地理空间信息学院,河南 郑州 450001
    2.信息工程大学数据与目标工程学院,河南 郑州 450001
  • 收稿日期:2025-07-17 修回日期:2025-11-13 出版日期:2026-01-15 发布日期:2026-01-15
  • 通讯作者: 冯凡 E-mail:zhangjrs0802@163.com;fengrs1991@163.com
  • 作者简介:张津(1994—),女,博士生,主要研究方向为深度学习、遥感图像分类。 E-mail:zhangjrs0802@163.com
  • 基金资助:
    国家自然科学基金(42071340);嵩山实验室项目(纳入河南省重大科技专项管理体系)(221100211000-04)

Small-sample classification of hyperspectral images based on mixed CNN-ViT feature optimization

Jin ZHANG1(), Fan FENG1(), Chenguang DAI1, Zhenchao ZHANG1, Ying YU1, Bing LIU2   

  1. 1.Institute of Geospatial Information, Information Engineering University, Zhengzhou 450001, China
    2.Institute of Data and Target Engineering, Information Engineering University, Zhengzhou 450001, China
  • Received:2025-07-17 Revised:2025-11-13 Online:2026-01-15 Published:2026-01-15
  • Contact: Fan FENG E-mail:zhangjrs0802@163.com;fengrs1991@163.com
  • About author:ZHANG Jin (1994—), female, PhD candidate, majors in deep learning and remote sensing image classification. E-mail: zhangjrs0802@163.com
  • Supported by:
    The National Natural Science Foundation of China(42071340);Program of Song Shan Laboratory (Included in the management of Major Science and Technology Program of Henan Province)(221100211000-04)

摘要:

高光谱图像分类是实现地物要素精细识别的关键技术。随着成像技术发展,无人机平台获取的高光谱图像空间分辨率不断提升,给地物精细分类带来了新的机遇和挑战。现有深层网络在小样本条件下对高空间分辨率高光谱图像特征学习不全面。针对以上问题,本文提出了一种针对卷积神经网络(CNN)和ViT混合特征的优化方法,包括自适应空谱特征学习、双向特征整合和多段特征交互增强3个方面。首先,将多尺度3D空谱特征和局部2D自注意力特征纳入级联残差结构,完成全局-局部多尺度空谱特征提取,增强特征的丰富性。然后,从两个方向整合空间特征和通道特征,提取两个维度的相关性,实现对CNN和ViT提取特征的补充和增强。将上述多段特征融合后,输入分解二阶池化层,解决多段特征之间差异大、缺乏交互的问题。最后,将细粒度融合特征输入全连接层,完成分类。在3个高空间分辨率高光谱图像数据集LongKou、HanChuan、HongHu上进行了小样本分类试验。每类地物仅使用5个样本训练模型,本文方法分类精度分别为94.00%、83.24%和87.63%,验证了本文方法在小样本条件下的有效性。

关键词: 高光谱图像分类, 混合卷积网络, 局部自注意力, 分解二阶池化, 多特征优化, 小样本

Abstract:

Hyperspectral image classification is a key technology for achieving fine-grained recognition of ground objects. With the advancement of imaging technology, the spatial resolution of hyperspectral images acquired by UAV platforms has significantly improved, bringing new opportunities and challenges to fine-grained land cover classification. However, existing deep neural networks still exhibit insufficiently comprehensive feature learning for high spatial-resolution hyperspectral images under small-sample conditions. To address this issue, this paper proposes a mixed feature optimization method of convolutional neural networks (CNN) and vision Transformer (ViT), including three aspects: adaptive spatial-spectral feature learning, bidirectional feature integration and multi-segment feature interaction enhancement. First, multi-scale 3D spatial-spectral features and 2D local selfattention features are incorporated into a cascaded residual structure to achieve global-local multi-scale spatial-spectral feature extraction, enhancing the feature richness. Then, spatial and channel features are integrated from two directions to extract correlations across both dimensions, thereby complementing and enhancing the features extracted by CNN and ViT. After fusing these multi-stage features, they are fed into a factorized second order pooling layer to address the issues of large discrepancies and insufficient interaction among multi-stage features. Finally, the fine-grained fused features are input into a fully connected layer for classification. Small-sample classification experiments were conducted on three hyperspectral image datasets with high spatial resolution, namely LongKou, HanChuan, and HongHu. Only five samples per land-cover class are used for model training. The proposed method achieved classification accuracies of 94.00%, 83.24%, and 87.63%, respectively, demonstrating its effectiveness under small-sample conditions.

Key words: hyperspectral image classification, mixed convolutional network, local selfattention, factorized second order pooling, multi-feature optimization, small sample

中图分类号: