高分辨率遥感影像深度迁移可变形卷积的场景分类法

doi:10.11947/j.AGCS.2021.20200190

测绘学报 ›› 2021, Vol. 50 ›› Issue (5): 652-663.doi: 10.11947/j.AGCS.2021.20200190

高分辨率遥感影像深度迁移可变形卷积的场景分类法

施慧慧^1,2, 徐雁南^1,2, 滕文秀³, 王妮^4,5

1. 南京林业大学南方现代林业协同创新中心, 江苏南京 210037;
2. 南京林业大学林学院, 江苏南京 210037;
3. 马萨诸塞大学阿默斯特分校地球科学系, 美国马萨诸塞州 01003;
4. 安徽省地理信息智能感知与服务工程实验室, 安徽滁州 239000;
5. 滁州学院地理信息与旅游学院, 安徽滁州 239000

收稿日期:2020-05-18 修回日期:2021-02-02 发布日期:2021-06-03
通讯作者: 徐雁南 E-mail:nfuxyn@njfu.edu.cn
作者简介:施慧慧(1996-),女,硕士生,主要研究方向为遥感影像信息提取与机器学习。E-mail:njfushihuihui@163.com
基金资助:
国家重点研发计划（2019YFD1100404）

Scene classification of high-resolution remote sensing imagery based on deep transfer deformable convolutional neural networks

SHI Huihui^1,2, XU Yannan^1,2, TENG Wenxiu³, WANG Ni^4,5

1. Co-Innovation Center for the Sustainable Forestry in Southern China, Nanjing Forestry University, Nanjing 210037, China;
2. College of Forest, Nanjing Forestry University, Nanjing 210037, China;
3. Department of Geosciences, University of Massachusetts, Massachusetts 01003, USA;
4. Anhui Engineering Laboratory of Geographical Information Intelligent Sensor and Service, Chuzhou 239000, China;
5. School of Geographic Information and Tourism, Chuzhou University, Chuzhou 239000, China

Received:2020-05-18 Revised:2021-02-02 Published:2021-06-03
Supported by:
The National Key Research and Development Program of China (No. 2019YFD1100404)

摘要/Abstract

摘要： 近年来基于深度卷积神经网络的高分辨率遥感影像场景分类成为广泛关注的焦点。由于现有深度卷积神经网络对遥感场景影像的几何形变不具有稳健性，本文提出了一种基于深度迁移可变形卷积神经网络（DTDCNN）的场景分类方法。该方法先利用大型自然场景数据集ImageNet上训练的深度模型提取遥感影像的深度特征，然后引入可变形卷积层，进一步学习对遥感场景的几何形变具有稳健性的深度特征。结果表明：增加可变形卷积后，DTDCNN在AID、UC-Merced和NWPU-RESISC45数据集上的精度分别提高了4.25%、1.9%和4.83%。该方法通过对场景中不同目标进行感受野自适应调整，增强了空间采样位置能力，有效提高了遥感场景分类的精度。

关键词: 遥感, 场景分类, 卷积神经网络, 可变形卷积, 迁移学习

Abstract: In recent years, scene classification of high-resolution remote sensing images based on deep convolutional neural networks has become the focus of attention. Because of the existing deep convolution neural network is not robust to the geometric deformation of remote sensing scene image, we proposed a novel scene classification method for high-resolution remote sensing image, based on the deep transfer deformable convolutional neural networks (DTDCNN). Specifically, the depth features of remote sensing image are extracted by using the trained depth model on the large-scale natural scene dataset (ImageNet), then, the deformable convolution layer is introduced to learn the depth features which are robust to the geometric deformation of remote sensing scene.The results show that:the accuracy of DTDCNN on AID, UC-Merced and NWPU-RESISC45 datasets is improved by 4.25%, 1.9% and 4.83% after adding the deformable convolution, respectively. By the adaptive adjustment of the receptive field for different objects in the scene, DTDCNN enhances the ability of spatial sampling position, and, effectively improves the accuracy of remote sensing scene classification.

Key words: remote sensing, scene classification, convolutional neural networks, deformable convolutional, transfer learning

中图分类号:

P237

施慧慧, 徐雁南, 滕文秀, 王妮. 高分辨率遥感影像深度迁移可变形卷积的场景分类法[J]. 测绘学报, 2021, 50(5): 652-663.

SHI Huihui, XU Yannan, TENG Wenxiu, WANG Ni. Scene classification of high-resolution remote sensing imagery based on deep transfer deformable convolutional neural networks[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(5): 652-663.

参考文献

[1] 李德仁, 童庆禧, 李荣兴, 等. 高分辨率对地观测的若干前沿科学问题[J]. 中国科学(地球科学), 2012, 42(6):805-813. LI Deren, TONG Qingxi, LI Rongxing, et al. Current issues in high-resolution Earth observation technology[J]. Science China Earth Sciences, 2012, 42(6):805-813.
[2] YANG Yi, NEWSAM S. Bag-of-visual-words and spatial extensions for land-use classification[C]//Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems.[S.l.]:ACM, 2010:270-279.
[3] CHERIYADAT A M. Unsupervised feature learning for aerial scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2014, 52(1):439-451.
[4] 张帆. 面向高分辨率遥感影像分析的深度学习方法研究[D]. 武汉:武汉大学, 2017. ZHANG Fan. Deep learning for very high resolution remote sensing data analysis[D]. Wuhan:Wuhan University, 2017.
[5] ZHU Qiqi, ZHONG Yanfei, ZHAO Bei, et al. Bag-of-Visual-Words scene classifier with local and global features for high spatial resolution remote sensing imagery[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(6):747-751.
[6] 钱晓亮, 李佳, 程塨, 等. 特征提取策略对高分辨率遥感图像场景分类性能影响的评估[J]. 遥感学报, 2018, 22(5):758-776. QIAN Xiaoliang, LI Jia, CHENG Gong, et al. Evaluation of the effect of feature extraction strategy on the performance of high-resolution remote sensing image scene classification[J]. Journal of Remote Sensing, 2018, 22(5):758-776.
[7] 闫苗, 赵红东, 李宇海, 等. 基于卷积神经网络的高光谱遥感地物多分类识别[J]. 激光与光电子学进展, 2019, 56(2):021702. YAN Miao, ZHAO Hongdong, LI Yuhai, et al. Multi-classification and recognition of hyperspectral remote sensing objects based on convolutional neural network[J]. Laser & Optoelectronics Progress, 2019, 56(2):021702.
[8] 龚希, 吴亮, 谢忠, 等. 融合全局和局部深度特征的高分辨率遥感影像场景分类方法[J]. 光学学报, 2019, 39(3):0301002. GONG Xi, WU Liang, XIE Zhong, et al. Classification method of high-resolution remote sensing scenes based on fusion of global and local deep features[J]. Acta Optica Sinica, 2019, 39(3):0301002.
[9] MARMANIS D, DATCU M, ESCH T, et al. Deep learning earth observation classification using ImageNet pretrained networks[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(1):105-109.
[10] 庄福振,罗平,何清,等.远移学习研究进展[J].软件学报,2015,26(1):26-39. ZHUANG Fuzhen, LUO Ping, HE Qing, et al. Survey on transter learning research[J]. Journal of Software, 2015, 26(1):26-39.
[11] DENG Jia, DONG Wei, SOCHER R, et al. ImageNet:a large-Scale hierarchical image database[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2009). Miami, FL:IEEE, 2009.
[12] 谭琨, 王雪, 杜培军. 结合深度学习和半监督学习的遥感影像分类进展[J]. 中国图象图形学报, 2019, 24(11):1823-1841. TAN Kun, WANG Xue, DU Peijun. Research progress of the remote sensing classification combining deep learning and semi-supervised learning[J]. Journal of Image and Graphics, 2019, 24(11):1823-1841.
[13] PENATTI O A B, NOGUEIRA K, SANTOS J A D. Do deep features generalize from everyday objects to remote sensing and aerial scenes domains?[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Boston, MA:IEEE, 2015.
[14] BASU S, GANGULY S, MUKHOPADHYAY S, et al. DeepSat:a learning framework for satellite imagery[C]//Proceedings of the 23rd SIGSPATIAL International Conference on Advances in Geographic Information Systems.[S.l.]:ACM, 2015.
[15] 蔡博文, 王树根, 王磊, 等. 基于深度学习模型的城市高分辨率遥感影像不透水面提取[J]. 地球信息科学学报, 2019, 21(9):1420-1429. CAI Bowen, WANG Shugen, WANG Lei, et al. Extraction of urban impervious surface from high-resolution remote sensing imagery based on deep learning[J]. Journal of Geo-information Science, 2019, 21(9):1420-1429.
[16] DAI Jifeng, QI Haozhi, XIONG Yuwen, et al. Deformable convolutional networks[C]//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy:IEEE, 2017.
[17] ZUO Zongcheng, ZHANG Wen, ZHANG Dongying. A remote sensing image semantic segmentation method by combining deformable convolution with conditional random fields[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(3):39-49. DOI:10.11947/j.JGGS.2020.0304.
[18] ZHANG Wei, TANG Ping, ZHAO Lijun. Remote sensing image scene classification using CNN-CapsNet[J]. Remote Sensing, 2019, 11(5):494.
[19] QI Kunlun, GUAN Qingfeng, YANG Chao, et al. Concentric circle pooling in deep convolutional networks for remote sensing scene classification[J]. Remote Sensing, 2018, 10(6):934.
[20] GONG Jianya, JI Shunping. Photogrammetry and deep learning[J]. Journal of Geodesy and Geoinformation Science, 2018, 1(1):1-15. DOI:10.11947/j.JGGS.2018.0101.
[21] 滕文秀, 温小荣, 王妮, 等. 基于深度迁移学习的无人机高分影像树种分类与制图[J]. 激光与光电子学进展, 2019, 56(7):072801. TENG Wenxiu, WEN Xiaorong, WANG Ni, et al. Tree species classification and mapping based on deep transfer learning with unmanned aerial vehicle high resolution images[J]. Laser & Optoelectronics Progress, 2019, 56(7):072801.
[22] PAN S J, YANG Qiang. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10):1345-1359.
[23] ZHU Xizhou, HU Han, LIN S, et al. Deformable ConvNets V2:more deformable, better results[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA:IEEE, 2019:9308-9316.
[24] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV:IEEE, 2016.
[25] 叶利华, 王磊, 张文文, 等. 高分辨率光学遥感场景分类的深度度量学习方法[J]. 测绘学报, 2019, 48(6):698-707. DOI:10.11947/j.AGCS.2019.20180434. YE Lihua, WANG Lei, ZHANG Wenwen, et al. Deep metric learning method for high resolution remote sensing image scene classification[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6):698-707. DOI:10.11947/j.AGCS.2019.20180434.
[26] XIA Guisong, HU Jingwen, HU Fan, et al. AID:a benchmark data set for performance evaluation of aerial scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7):3965-3981.
[27] 滕文秀, 王妮, 陈泰生, 等. 基于深度对抗域适应的高分辨率遥感影像跨域分类[J]. 激光与光电子学进展, 2019, 56(11):112801. TENG Wenxiu, WANG Ni, CHEN Taisheng, et al. Deep adversarial domain adaptation method for cross-domain classification in high-resolution remote sensing images[J]. Laser & Optoelectronics Progress, 2019, 56(11):112801.
[28] CHENG Gong, HAN Junwei, LU Xiaoqiang. Remote sensing image scene classification:benchmark and state of the art[J]. Proceedings of the IEEE, 2017, 105(10):1865-1883.
[29] WEI Yufan, LUO Xiaobo, HU Lixin, et al. An improved unsupervised representation learning generative adversarial network for remote sensing image scene classification[J]. Remote Sensing Letters, 2020, 11(6):598-607.
[30] LIU Xuning, ZHOU Yong, ZHAO Jiaqi, et al. Siamese convolutional neural networks for remote sensing scene classification[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(8):1200-1204.
[31] CHAIB S, LIU Huan, GU Yanfeng, et al. Deep feature fusion for VHR remote sensing scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(8):4775-4784.
[32] WANG Guoli, FAN Bin, XIANG Shiming, et al. Aggregating rich hierarchical features for scene classification in remote sensing imagery[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2017, 10(9):4104-4115.
[33] HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI:IEEE, 2018.
[34] WANG Chen, LIN Wei, TANG Pengfei. Multiple resolution block feature for remote-sensing scene classification[J]. International Journal of Remote Sensing, 2019, 40(17-18):6884-6904.
[35] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]//Proceedings of International Conference on Learning Representations. San Diego:ICLR, 2015:1-5.
[36] WANG Yuliang, LI Mingshi. Urban impervious surface detection from remote sensing images:a review of the methods and challenges[J]. IEEE Geoscience and Remote Sensing Magazine, 2019, 7(3):64-93.
[37] WANG Yuliang, SU Huiyi, LI Ming. An improved model based detection of urban impervious surfaces using multiple features extracted from Rosis-3 hyperspectral images[J]. Remote Sensing, 2019, 11(2):136.

高分辨率遥感影像深度迁移可变形卷积的场景分类法

Scene classification of high-resolution remote sensing imagery based on deep transfer deformable convolutional neural networks

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	张祖勋, 姜慧伟, 庞世燕, 胡翔云. 多时相遥感影像的变化检测研究现状与展望[J]. 测绘学报, 2022, 51(7): 1091-1107.
[2]	王桥. 地表异常遥感探测与即时诊断方法研究框架[J]. 测绘学报, 2022, 51(7): 1141-1152.
[3]	张克非, 李浩博, 王晓明, 朱丹彤, 何琦敏, 李龙江, 胡安东, 郑南山, 李怀展. 地基GNSS大气水汽探测遥感研究进展和展望[J]. 测绘学报, 2022, 51(7): 1172-1191.
[4]	张良培, 何江, 杨倩倩, 肖屹, 袁强强. 数据驱动的多源遥感信息融合研究进展[J]. 测绘学报, 2022, 51(7): 1317-1337.
[5]	吴立新, 齐源, 毛文飞, 刘善军, 丁逸凡, 荆凤, 申旭辉. 多波段多极化被动微波遥感地震应用研究进展与前沿方向探索[J]. 测绘学报, 2022, 51(7): 1356-1371.
[6]	张兵, 杨晓梅, 高连如, 孟瑜, 孙显, 肖晨超, 倪丽. 遥感大数据智能解译的地理学认知模型与方法[J]. 测绘学报, 2022, 51(7): 1398-1415.
[7]	许强, 朱星, 李为乐, 董秀军, 戴可人, 蒋亚楠, 陆会燕, 郭晨. “天-空-地”协同滑坡监测技术进展[J]. 测绘学报, 2022, 51(7): 1416-1436.
[8]	李德仁, 王密, 杨芳. 新一代智能测绘遥感科学试验卫星珞珈三号01星[J]. 测绘学报, 2022, 51(6): 789-796.
[9]	龚健雅, 宦麟茜, 郑先伟. 影像解译中的深度学习可解释性分析方法[J]. 测绘学报, 2022, 51(6): 873-884.
[10]	张勤, 赵超英, 陈雪蓉. 多源遥感地质灾害早期识别技术进展与发展趋势[J]. 测绘学报, 2022, 51(6): 885-896.
[11]	李荣兴, 李国君, 冯甜甜, 沈强, 乔刚, 叶真, 夏梦莲. 基于光学遥感卫星影像的南极冰流速产品和方法研究综述[J]. 测绘学报, 2022, 51(6): 953-963.
[12]	程结海, 黄中意, 王建如, 何湜. 高空间分辨率遥感影像最优分割结果自动确定方法[J]. 测绘学报, 2022, 51(5): 658-667.
[13]	白坤, 慕晓冬, 陈雪冰, 朱永清, 尤轩昂. 融合半监督学习的无监督遥感影像场景分类[J]. 测绘学报, 2022, 51(5): 691-702.
[14]	龚健雅, 张觅, 胡翔云, 张展, 李彦胜, 姜良存. 智能遥感深度学习框架与模型设计[J]. 测绘学报, 2022, 51(4): 475-487.
[15]	童小华, 刘世杰, 谢欢, 许雄, 叶真, 冯永玖, 王超, 柳思聪, 金雁敏, 陈鹏, 洪中华, 栾奎峰. 从地球测绘到地外天体测绘[J]. 测绘学报, 2022, 51(4): 488-500.