一种耦合DeepLab与Transformer的农作物种植类型遥感精细分类方法

doi:10.11947/j.AGCS.2024.20220692

测绘学报 ›› 2024, Vol. 53 ›› Issue (2): 353-366.doi: 10.11947/j.AGCS.2024.20220692

一种耦合DeepLab与Transformer的农作物种植类型遥感精细分类方法

林云浩^1,2,3, 王艳军^1,2,3, 李少春^1,2,3, 蔡恒藩^1,2,3

1. 湖南科技大学测绘遥感信息工程湖南省重点实验室, 湖南湘潭 411201;
2. 湖南科技大学地理空间信息技术国家地方联合工程实验室, 湖南湘潭 411201;
3. 湖南科技大学地球科学与空间信息工程学院, 湖南湘潭 411201

收稿日期:2022-12-19 修回日期:2023-09-05 发布日期:2024-03-08
通讯作者: 王艳军 E-mail:wongyanjun@163.com
作者简介:林云浩(1999-),男,硕士生,研究方向为多源遥感数据处理。E-mail:1056267419@qq.com
基金资助:
国家自然科学基金（41971423；31972951）

A coupled DeepLab and Transformer approach for fine classification of crop cultivation types in remote sensing

LIN Yunhao^1,2,3, WANG Yanjun^1,2,3, LI Shaochun^1,2,3, CAI Hengfan^1,2,3

1. Hunan Provincial Key Laboratory of Geo-Information Engineering in Surveying, Mapping and Remote Sensing, Hunan University of Science and Technology, Xiangtan 411201, China;
2. National-local Joint Engineering Laboratory of Geo-spatial Information Technology, Hunan University of Science and Technology, Xiangtan 411201, China;
3. School of Earth Sciences and Spatial Information Engineering, Hunan University of Science and Technology, Xiangtan 411201, China

Received:2022-12-19 Revised:2023-09-05 Published:2024-03-08
Supported by:
The National Natural Science Foundation of China (Nos. 41971423; 31972951)

摘要/Abstract

摘要： 如何精细遥感监测复杂的不同类型农田作物种植情况，是智慧农业农村领域实现农耕面积调查与农作物估产的关键。目前的高分辨率影像的作物种植像素级语义分割中，深度卷积神经网络难以兼顾空间多尺度全局特征和局部细节特征，从而导致各类农田地块之间边界轮廓模糊和同类农田区域内部完整性不高等问题。针对这些不足，本文提出了一种耦合DeepLabv3+和Transformer编码器的双分支并行特征融合网络FDTNet，以实现农作物种植类型的精细遥感监测。首先，在FDTNet中并行嵌入DeepLabv3+和Transformer分别捕获农田影像的局部特征和全局特征；其次，应用耦合注意力融合模块CAFM有效融合两者的特征；然后，在解码器阶段应用卷积注意力模块CBAM增强卷积层有效特征的权重；最后，采用渐进式多层特征融合策略将编码器和解码器中的有效特征全面融合并输出特征图，以实现晚稻、中稻、藕田、菜地和大棚的高精度分类识别。为了验证FDTNet网络模型在高分辨率作物分类应用的有效性，本文选择不同高分辨率的Yuhu数据集和Zhejiang数据集验证，mIoU分别达到74.7%和81.4%。相比于已有的UNet、DeepLabv3、DeepLabv3+、ResT和Res-Swin等深度学习方法，FDTNet的mIoU可分别高2.2%和3.6%。结果表明，FDTNet在纹理单一、大样本量，以及纹理多样、小样本量的两类农田场景中同时表现出优于对比方法的性能，具有较全面的多类别农作物有效特征提取能力。

关键词: 高分辨率遥感影像, 农作物种植类型, 语义分割, 特征融合, 深度学习

Abstract: How to accurately monitor the planting of different types of complex farmland crops by remote sensing is the key to the realization of agricultural area survey and crop yield estimation in the area of smart rural agriculture. In the current pixel level semantic segmentation of crop planting in high-resolution images, the deep convolution neural network is difficult to take into account the spatial multi-scale global features and local details, which leads to problems such as blurring boundary contours between various farmland plots and low internal integrity of the same farmland area. In view of these shortcomings, this paper designs and proposes a dual branch parallel feature fusion network (FDTNet) that couples DeepLabv3+and Transformer encoders to achieve fine remote sensing monitoring of crop planting. Firstly, DeepLabv3+and Transformer are embedded in FDTNet in parallel to capture the local and global features of farmland image respectively. Secondly, the coupled attention fusion module (CAFM) is used to effectively fuse the characteristics of the two features. Then, in the decoder stage, the convolutional block attention module (CBAM) is applied to enhance the weight of the effective features of the convolutional layer. Finally, the progressive multi-level feature fusion strategy is adopted to fully fuse the effective features in the encoder and deco-der, and output the feature map to achieve high-precision classification and recognition of late rice, middle rice, lotus root field, vegetable field and greenhouse. In order to verify the effectiveness of FDTNet network model in high-resolution crop classification application, this paper selects different high-resolution Yuhu dataset and Zhejiang dataset and experimental results of mIoU reach 74.7% and 81.4%, respectively. The mIoU of FDTNet can be 2.2% and 3.6% respectively higher than the existing deep learning methods, such as UNet, DeepLabv3, DeepLabv3+, ResT and Res-Swin. The results show that FDTNet has better classification performance than the compared methods in two types of farmland scenes, which have single texture and large sample size, or multiple texture and small sample size. The proposed FDTNet has a comprehensive ability to extract effective features of multiple category crops.

Key words: high-resolution remote sensing image, crop planting type, semantic segmentation, feature fusion, deep learning

中图分类号:

P237

林云浩, 王艳军, 李少春, 蔡恒藩. 一种耦合DeepLab与Transformer的农作物种植类型遥感精细分类方法[J]. 测绘学报, 2024, 53(2): 353-366.

LIN Yunhao, WANG Yanjun, LI Shaochun, CAI Hengfan. A coupled DeepLab and Transformer approach for fine classification of crop cultivation types in remote sensing[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(2): 353-366.

参考文献

[1] ADÃO T, HRUŠKA J, PÁDUA L, et al. Hyperspectral imaging:a review on UAV-based sensors, data processing and applications for agriculture and forestry[J]. Remote Sensing, 2017, 9(11):1110.
[2] ZHONG Yanfei, WANG Xinyu, XU Yao, et al. Mini-UAV-borne hyperspectral remote sensing:from observation and processing to applications[J]. IEEE Geoscience and Remote Sensing Magazine, 2018, 6(4):46-62.
[3] 刘海启. 以精准农业驱动农业现代化加速现代农业数字化转型[J]. 中国农业资源与区划, 2019, 40(1):1-6, 73. LIU Haiqi. Accelerating the digital transformation of modern agriculture by driving the agricultural modernization with precision agriculture[J]. Chinese Journal of Agricultural Resources and Regional Planning, 2019, 40(1):1-6, 73.
[4] 唐华俊, 吴文斌, 杨鹏, 等. 农作物空间格局遥感监测研究进展[J]. 中国农业科学, 2010, 43(14):2879-2888. TANG Huajun, WU Wenbin, YANG Peng, et al. Recent progresses in monitoring crop spatial patterns by using remote sensing technologies[J]. Scientia Agricultura Sinica, 2010, 43(14):2879-2888.
[5] 赵春江. 农业遥感研究与应用进展[J]. 农业机械学报, 2014, 45(12):277-293. ZHAO Chunjiang. Advances of research and application in remote sensing for agriculture[J]. Transactions of the Chinese Society for Agricultural Machinery, 2014, 45(12):277-293.
[6] ZHONG Yanfei, HU Xin, LUO Chang, et al. WHU-Hi:UAV-borne hyperspectral with high spatial resolution (H²) benchmark datasets and classifier for precise crop identification based on deep convolutional neural network with CRF[J]. Remote Sensing of Environment, 2020, 250:112012.
[7] 刘巍, 吴志峰, 骆剑承, 等. 深度学习支持下的丘陵山区耕地高分辨率遥感信息分区分层提取方法[J]. 测绘学报, 2021, 50(1):105-116. DOI:10.11947/j.AGCS.2021.20190448. LIU Wei, WU Zhifeng, LUO Jiancheng, et al. A divided and stratified extraction method of high-resolution remote sensing information for cropland in hilly and mountainous areas based on deep learning[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(1):105-116.DOI:10.11947/j.AGCS.2021.20190448.
[8] 叶思菁. 大数据环境下遥感图谱应用方法研究:以作物干旱监测为例[J]. 测绘学报, 2018, 47(6):892.DOI:10.11947/j.AGCS.2018.20170535. YE Sijing. Research on application of remote sensing tupu-take monitoring of meteorological disaster for example[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6):892.DOI:10.11947/j.AGCS.2018.20170535.
[9] CHAMORRO M J A, CUÉ LA ROSA L E, FEITOSA R Q, et al. Fully convolutional recurrent networks for multidate crop recognition from multitemporal image sequences[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 171:188-201.
[10] 陈劲松, 黄健熙, 林珲, 等. 基于遥感信息和作物生长模型同化的水稻估产方法研究[J]. 中国科学:信息科学, 2010, 40(S1):173-183. CHEN Jinsong, HUANG Jianxi, LIN Hui, et al.Study on rice yield estimation method based on assimilation of remote sensing information and crop growth model[J]. Scientia Sinica (Informationis), 2010, 40(S1):173-183.
[11] DE LA CASA A, OVANDO G, BRESSANINI L, et al. Soybean crop coverage estimation from NDVI images with different spatial resolution to evaluate yield variability in a plot[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 146:531-547.
[12] BEERI O, PELED A. Geographical model for precise agriculture monitoring with real-time remote sensing[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2009, 64(1):47-54.
[13] ASHAPURE A, JUNG J, YEOM J, et al. A novel framework to detect conventional tillage and no-tillage cropping system effect on cotton growth and development using multi-temporal UAS data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 152:49-64.
[14] 蒙继华, 吴炳方, 杜鑫, 等. 遥感在精准农业中的应用进展及展望[J]. 国土资源遥感, 2011, 23(3):1-7. MENG Jihua, WU Bingfang, DU Xin, et al. A review and outlook of applying remote sensing to precision agriculture[J]. Remote Sen-sing for Land & Resources, 2011, 23(3):1-7.
[15] 李德仁, 童庆禧, 李荣兴, 等. 高分辨率对地观测的若干前沿科学问题[J]. 中国科学:地球科学, 2012, 42(6):805-813. LI Deren, TONG Qingxi, LI Rongxing, et al.Some frontier scientific problems of high-resolution earth observation[J]. Scientia Sinica (Terrae), 2012, 42(6):805-813.
[16] LICHTBLAU E, OSWALD C J. Classification of impervious land-use features using object-based image analysis and data fusion[J]. Computers, Environment and Urban Systems, 2019, 75:103-116.
[17] SALMAN N H, LIU C Q. Image segmentation and edge detection based on watershed techniques[J]. International Journal of Compu-ters and Applications, 2003, 25(4):258-263.
[18] RYDBERG A, BORGEFORS G. Integrated method for boundary delineation of agricultural fields in multispectral satellite images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2001, 39(11):2514-2520.
[19] MOUNTRAKIS G, IM J, OGOLE C. Support vector machines in remote sensing:a review[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2011, 66(3):247-259.
[20] PIIROINEN R, HEISKANEN J, MÕTTUS M, et al. Classification of crops across heterogeneous agricultural landscape in Kenya using Aisa EAGLE imaging spectroscopy data[J]. International Journal of Applied Earth Observation and Geoinformation, 2015, 39:1-8.
[21] BELGIU M, DRǍGUŢ L. Random forest in remote sensing:a review of applications and future directions[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 114:24-31.
[22] ZHONG Yanfei, LIN Xuemei, ZHANG Liangpei. A support vector conditional random fields classifier with a mahalanobis distance boundary constraint for high spatial resolution remote sensing imagery[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(4):1314-1330.
[23] ZHAO Ji, ZHONG Yanfei, ZHANG Liangpei. Detail-preserving smoothing classifier based on conditional random fields for high spatial resolution remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(5):2440-2452.
[24] ZHAO Ji, ZHONG Yanfei, JIA Tianyi, et al. Spectral-spatial classification of hyperspectral imagery with cooperative game[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 135:31-42.
[25] WANG Hong, CHEN Xianzhong, ZHANG Tianxiang, et al. CCTNet:coupled CNN and transformer network for crop segmentation of remote sensing images[J]. Remote Sensing, 2022, 14(9):1956.
[26] WANG Wang, LI Shaochun, WANG Wang, et al. A simple deep learning network for classification of 3D mobile LiDAR point clouds[J]. Journal of Geodesy and Geoinformation Science, 2021, 4(3):49-59.
[27] YANG S, GU L, LI X, et al. Crop classification method based on optimal feature selection and hybrid CNN-RF networks for multi-temporal remote sensing imagery[J]. Remote sensing, 2020, 12(19):3119.
[28] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA:IEEE, 2016:770-778.
[29] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston:IEEE, 2015:3431-3440.
[30] 胡明洪, 李佳田, 姚彦吉, 等. 结合多路径的高分辨率遥感影像建筑物提取SER-UNet算法[J]. 测绘学报, 2023, 52(5):808-817. DOI:10.11947/j.AGCS.2023.20210691. HU Minghong, LI Jiatian, YAO Yanji, et al. SER-UNet algorithm for building extraction from high-resolution remote sensing image combined with multipath[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(5):807-817, DOI:10.11947/j.AGCS.2023.20210691.
[31] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas:IEEE, 2016:770-778.
[32] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab:semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):834-848.
[33] REN Y, ZHANG X, MA Y, et al. Full convolutional neural network based on multi-scale feature fusion for the class imbalance remote sensing image classification[J]. Remote Sensing, 2020, 12(21):3547.
[34] CHEN L C, ZHU Yukun, PAPANDREOU G, et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of 2018 Computer Vision European Conference. Munich:ACM Press, 2018:833-851.
[35] 许泽宇, 沈占锋, 李杨, 等. 增强型DeepLab算法和自适应损失函数的高分辨率遥感影像分类[J]. 遥感学报, 2022, 26(2):406-415. XU Zeyu, SHEN Zhanfeng, LI Yang, et al. Classification of high-resolution remote sensing images based on enhanced Deep Lab algorithm and adaptive loss function[J]. National Remote Sensing Bulletin, 2022, 26(2):406-415.
[36] REEDHA R, DERICQUEBOURG E, CANALS R, et al. Transformer neural network for weed and crop classification of high resolution UAV images[J]. Remote Sensing, 2022, 14(3):592.
[37] YUAN Li, HOU Qibin, JIANG Zihang, et al. VOLO:vision outlooker for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(5):6575-6586.
[38] LIU Ze, LIN Yutong, CAO Yue, et al. Swin Transformer:hierarchical Vision Transformer using Shifted Windows[C]//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal:IEEE, 2021:9992-10002.
[39] DONG Xiaoyi, BAO Jianmin, CHEN Dongdong, et al. CSWin transformer:a general vision transformer backbone with cross-shaped windows[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans:IEEE, 2022:12114-12124.
[40] XIAO T, SINGH M, MINTUN E, et al. Early convolutions help transformers see better[J]. Advances in Neural Information Processing Systems, 2021, 34:30392-30400.
[41] ZHANG Q, YANG Y B. ResT:an efficient transformer for visual recognition[J]. Advances in Neural Information Processing Systems, 2021, 34:15475-15485.
[42] SRINIVAS A, LIN T Y, PARMAR N, et al. Bottleneck transformers for visual recognition[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville:IEEE, 2021:16514-16524.
[43] DAI Z, LIU H, LE Q V, et al. Coatnet:marrying convolution and attention for all data sizes[J]. Advances in Neural Information Processing Systems, 2021, 34:3965-3977.
[44] NIU B, FENG Q, CHEN B, et al. HSI-TransUNet:a transformer based semantic segmentation model for crop mapping from UAV hyperspectral imagery[J]. Computers and Electronics in Agriculture, 2022, 201:107297.
[45] PENG Zhiliang, HUANG Wei, GU Shanzhi, et al. Conformer:local features coupling global representations for visual recognition[C]//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal:IEEE, 2021:357-366.
[46] ZHANG Yundong, LIU Huiye, HU Qiang. TransFuse:fusing transformers and CNNs for medical image segmentation[M]//Medical Image Computing and Computer Assisted Intervention-MICCAI 2021. Cham:Springer International Publishing, 2021:14-24.
[47] DING L, LIN D, LIN Shaofu, et al. Looking outside the window:wide-context transformer for the semantic segmentation of high-resolution remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60:1-13.
[48] WOO S, PARK J, LEE J Y, et al. CBAM:convolutional block attention module[C]//Proceedings of 2018 European Conference on Computer Vision. Cham:Springer, 2018:3-19.
[49] ZHOU Lichen, ZHANG Chuang, WU Ming. D-LinkNet:LinkNet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City:IEEE, 2018:192-1924.
[50] ZHANG Cheng, JIANG Wanshou, ZHANG Yuan, et al. Transformer and CNN hybrid deep neural network for semantic segmentation of very-high-resolution remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:4408820.
[51] QIAN Z, CAO Y, SHI Z, et al. A semantic segmentation method for remote sensing images based on Deeplab v3[C]//Proceedings of 2021 International Conference on Big Data & Artificial Intelligence & Software Engineering. Zhuhai:IEEE, 2021:396-400.
[52] HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018:7132-7141.
[53] SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM:visual explanations from deep networks via gradient-based localization[C]//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). Venice:IEEE, 2017:618-626.

一种耦合DeepLab与Transformer的农作物种植类型遥感精细分类方法

A coupled DeepLab and Transformer approach for fine classification of crop cultivation types in remote sensing

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	廖钊宏, 张依晨, 杨飚, 林明春, 孙文博, 高智. 基于Swin Transformer-CNN的单目遥感影像高程估计方法及其在公路建设场景中的应用[J]. 测绘学报, 2024, 53(2): 344-352.
[2]	张彩丽, 向隆刚, 李雅丽, 高松峰, 潘传姣. 路段级导航属性信息挖掘[J]. 测绘学报, 2024, 53(2): 367-378.
[3]	江宝得, 黄威, 许少芬, 巫勇. 融合分散自适应注意力机制的多尺度遥感影像建筑物实例细化提取[J]. 测绘学报, 2023, 52(9): 1504-1514.
[4]	曹兴文, 郑宏伟, 刘英, 吴孟泉, 王灵玥, 包安明, 陈曦. 多行人轨迹多视角三维仿真视频学习预测法[J]. 测绘学报, 2023, 52(9): 1595-1608.
[5]	杜培军, 张伟, 张鹏, 林聪, 郭山川, 胡泽周. 一种联合空谱特征的高光谱影像分类胶囊网络[J]. 测绘学报, 2023, 52(7): 1090-1104.
[6]	余旭初, 刘冰, 薛志祥. 高光谱地物要素识别潜力分析与前景展望[J]. 测绘学报, 2023, 52(7): 1115-1125.
[7]	杭仁龙, 李成相, 刘青山. 谱间对比学习的高光谱图像无监督特征提取[J]. 测绘学报, 2023, 52(7): 1164-1174.
[8]	胡鑫, 王心宇, 钟燕飞. 基于自适应上下文聚合网络的双高遥感影像分类[J]. 测绘学报, 2023, 52(7): 1175-1186.
[9]	张蒙蒙, 李伟, 刘欢, 赵旭东, 陶然. 基于形态变换与空间逻辑聚合的高光谱森林树种分类[J]. 测绘学报, 2023, 52(7): 1202-1211.
[10]	宋尚真, 杨怡欣, 王会峰, 王晓艳, 荣生辉, 周慧鑫. 高光谱图像稀疏约束与自编码器特征提取相结合的异常检测方法[J]. 测绘学报, 2023, 52(6): 932-943.
[11]	胡功明, 杨春成, 徐立, 尚海滨, 王泽凡, 秦志龙. 改进U-Net的遥感图像语义分割方法[J]. 测绘学报, 2023, 52(6): 980-989.
[12]	宋佳璇, 范大昭, 董杨, 纪松, 李东子. 神经网络学习与灰度信息结合的跨视角影像线特征匹配算法[J]. 测绘学报, 2023, 52(6): 990-999.
[13]	邓敏, 罗斌, 唐建波, 姚志鹏, 刘国平, 温翔, 胡润波, 柴华, 胡文柯. 顾及轨迹密度分布异质性的道路交叉口提取方法[J]. 测绘学报, 2023, 52(6): 1000-1009.
[14]	马梦锴, 董箭, 唐露露, 彭认灿, 周寅飞, 王芳. 兼顾要素正确分类及精准定位的栅格海图水深注记自动提取方法[J]. 测绘学报, 2023, 52(6): 1022-1036.
[15]	胡明洪, 李佳田, 姚彦吉, 阿晓荟, 陆美, 李文. 结合多路径的高分辨率遥感影像建筑物提取SER-UNet算法[J]. 测绘学报, 2023, 52(5): 808-817.