深度卷积特征表达的多模态遥感影像模板匹配方法

doi:10.11947/j.AGCS.2019.20180432

测绘学报 ›› 2019, Vol. 48 ›› Issue (6): 727-736.doi: 10.11947/j.AGCS.2019.20180432

深度卷积特征表达的多模态遥感影像模板匹配方法

南轲, 齐华, 叶沅鑫

西南交通大学地球科学与环境工程学院, 四川成都 611756

收稿日期:2018-09-14 修回日期:2019-03-20 出版日期:2019-06-20 发布日期:2019-07-09
通讯作者: 叶沅鑫 E-mail:yeyuanxin@home.swjtu.edu.cn
作者简介:南轲(1994-),男,硕士生,研究方向为摄影测量与遥感。E-mail:nanke1994@163.com
基金资助:
四川省科技计划（2017SZ0027）

A template matching method of multimodal remote sensing images based on deep convolutional feature representation

NAN Ke, QI Hua, YE Yuanxin

Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 611756, China

Received:2018-09-14 Revised:2019-03-20 Online:2019-06-20 Published:2019-07-09
Supported by:
The Science and Technology Program of Sichuan Province(No. 2017SZ0027)

摘要/Abstract

摘要： 多模态遥感影像间（光学、红外、SAR等）存在显著的非线性辐射差异，传统方法难以有效地提取影像间的共有特征，匹配效果不佳。鉴于此，本文将深度学习方法引入影像匹配中，提出了一种基于Siamese网络提取多模态影像共有特征的匹配方法。首先通过去除Siamese网络中的池化层和抽取特征来优化该网络，保持特征信息的完整性和位置精度，使其可有效地提取多模态影像间的共有特征，然后采用模板匹配策略，实现多模态遥感影像高精度匹配。通过利用多组多模态遥感影像进行试验，结果表明，本文方法的匹配正确率和匹配精度都优于传统的模板匹配方法。

关键词: 多模态影像, 影像匹配, 深度学习, Siamese网络

Abstract: Due to significant non-linear radiometric differences between multimodal remote sensing images (e.g., optical, infrared, and SAR), traditional methods cannot efficiently extract common features between such images, and are vulnerable for image matching. To address that, the deep learning technique is introduced into the present study to design a matching method based on Siamese network, which aims to extract common features between multimodal images. The network is first optimized by removing the pooling layer and extracting the feature layer from Siamese network to maintain the integrity and positional accuracy of the feature information, making it possible the effective extraction of common features between multimodal images. Then, the template matching strategy is adopted to achieve high-precision matching of multimodal images. The proposed method is evaluated by using multiple multimodal remote sensing images. The results show that the proposed method outperforms traditional template-matching methods in both the matching correct ratio and matching accuracy.

Key words: multimodal image, image matching, deep learning, Siamese network

中图分类号:

P231

南轲, 齐华, 叶沅鑫. 深度卷积特征表达的多模态遥感影像模板匹配方法[J]. 测绘学报, 2019, 48(6): 727-736.

NAN Ke, QI Hua, YE Yuanxin. A template matching method of multimodal remote sensing images based on deep convolutional feature representation[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6): 727-736.

参考文献

[1] 闫利, 王紫琦, 叶志云. 顾及灰度和梯度信息的多模态影像配准算法[J]. 测绘学报, 2018, 47(1):71-81. DOI:10.11947/j.AGCS.2018.20170368. YAN Li, WANG Ziqi, YE Zhiyun. Multimodal image registration algorithm considering grayscale and gradient information[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(1):71-81. DOI:10.11947/j.AGCS.2018.20170368.
[2] YE Yuanxin, SHAN Jie, BRUZZONE L, et al. Robust registration of multimodal remote sensing images based on structural similarity[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(5):2941-2958.
[3] ZITOVÁ B, FLUSSER J. Image registration methods:a survey[J]. Image and Vision Computing, 2003, 21(11):977-1000.
[4] 余先川, 吕中华, 胡丹. 遥感图像配准技术综述[J]. 光学精密工程, 2013, 21(11):2960-2972. YU Xianchuan, LÜ Zhonghua, HU Dan. Review of remote sensing image registration techniques[J]. Optics and Precision Engineering, 2013, 21(11):2960-2972.
[5] MORAVEC H P. Towards automatic visual obstacle avoidance[C]//Proceedings of the 5th International Joint Conference on Artificial Intelligence. Massachusetts US:MIT, 1977.
[6] HARRIS C, STEPHENS M. A combined corner and edge detector[C]//Proceedings of the 4th Alvey Vision Conference. Manchester, UK:Organising Committee AVC, 1988:147-151.
[7] ZIOU D, TABBONE S. Edge detection techniques:an overview[J]. International Journal of Pattern Recognition and Image Analysis, 1998(8):537-559.
[8] ZHANG Dengrong, YU Le, CAI Zhigang. Automatic registration for ASAR and TM images based on region features[C]//Proceedings of SPIE 6752, Geoinformatics 2007:Remotely Sensed Data and Information. Nanjing:SPIE, 2007:6752.
[9] 张宝尚, 田铮, 延伟东. 基于分割区域的SAR图像配准方法研究[J]. 工程数学学报, 2011, 28(1):7-14. ZHANG Baoshang, TIAN Zheng, YAN Weidong. An SAR image registration algorithm based on segmentation-derived regions[J]. Chinese Journal of Engineering Mathematics, 2011, 28(1):7-14.
[10] LOWE D G. Object recognition from local scale-invariant features[C]//Proceeding of the 7th International Conference on Computer Vision. Kerkyra, Corfu, Greece:IEEE, 1999:1150-1157.
[11] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
[12] BELONGIE S, MALIK J, PUZICHA J. Shape matching and object recognition using shape contexts[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(4):509-522.
[13] SURI S, REINARTZ P. Mutual-information-based registration of TerraSAR-X and Ikonos imagery in urban areas[J]. IEEE Transactions on Geoscience and Remote Sensing, 2010, 48(2):939-949.
[14] MARTINEZ A, GARCIA-CONSUEGRA J, ABAD F. A correlation-symbolic approach to automatic remotely sensed image rectification[C]//Proceedings of IEEE 1999 International Geoscience and Remote Sensing Symposium. Hamburg, Germany:IEEE, 1999:336-338.
[15] JOHNSON K, COLE-RHODES A, ZAVORIN I, et al. Mutual information as a similarity measure for remote sensing image registration[C]//Proceedings of SPIE 4383, Geo-Spatial Image and Data Exploitation Ⅱ. Orlando, FL, United States:SPIE, 2001:51-61.
[16] YE Yuanxin, SHAN Jie, HAO Siyuan, et al. A local phase based invariant feature for remote sensing image matching[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018(142):205-221.
[17] ZAGORUYKO S, KOMODAKIS N. Learning to compare image patches via convolutional neural networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Boston:IEEE, 2015:4353-4361.
[18] 范大昭, 董杨, 张永生. 卫星影像匹配的深度卷积神经网络方法[J]. 测绘学报, 2018, 47(6):844-853. DOI:10.11947/j.AGCS.2018.20170627. FAN Dazhao, DONG Yang, ZHANG Yongsheng. Satellite image matching method based on deep convolution neural network[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6):844-853. DOI:10.11947/j.AGCS.2018.20170627.
[19] HE Haiqing, CHEN Min, CHEN Ting, et al. Matching of remote sensing images with complex background variations via Siamese convolutional neural network[J]. Remote Sensing, 2018, 10(3):355.
[20] BROMLEY J, GUYON I, LECUN Y, et al. Signature verification using a "Siamese" time delay neural network[C]//Proceedings of the 6th International Conference on Neural Information Processing Systems. Denver, Colorado:Morgan Kaufmann Publishers Inc, 1993.
[21] CHOPRA S, HADSELL R, LECUN Y. Learning a similarity metric discriminatively, with Application to Face Verification//Proceeding of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, CA:IEEE, 2005.
[22] 许慧敏. 基于深度学习U-Net模型的高分辨率遥感影像分类方法研究[D]. 成都:西南交通大学, 2018. XU Huimin. Method research of high resolution remote sensing imagery classification based on U-Net model of deep learning[D]. Chengdu:Southwest Jiaotong University, 2018.
[23] 周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6):1229-1251. ZHOU Feiyan, JIN Linpeng, DONG Jun. Review of convolutional neural network[J]. Chinese Journal of Computers, 2017,40(6):1229-1251.
[24] HADSELL R, CHOPRA S, LECUN Y. Dimensionality reduction by learning an invariant mapping[C]//Proceedins of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York:IEEE, 2006.
[25] TRAPPEY A J C, HSU F C, TRAPPEY C V, et al. Development of a patent document classification and search platform using a back-propagation network[J]. Expert Systems with Applications, 2006, 31(4):755-765.
[26] YE Yuanxin, SHAN Jie. A local descriptor based registration method for multispectral remote sensing images with non-linear intensity differences[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014(90):83-95.
[27] 林善明, 朱小艳, 周建华, 等. 一种结合互信息和模板匹配的配准方法[J]. 计算机工程, 2010, 36(14):198-200. LIN Shanming, ZHU Xiaoyan, ZHOU Jianhua, et al. Registration method with mutual information and template matching[J]. Computer Engineering, 2010, 36(14):198-200.

深度卷积特征表达的多模态遥感影像模板匹配方法

A template matching method of multimodal remote sensing images based on deep convolutional feature representation

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	张良培, 何江, 杨倩倩, 肖屹, 袁强强. 数据驱动的多源遥感信息融合研究进展[J]. 测绘学报, 2022, 51(7): 1317-1337.
[2]	程涛, 张洋, James Haworth. 基于网络和图的时空智能——概念、方法和应用[J]. 测绘学报, 2022, 51(7): 1629-1639.
[3]	龚健雅, 宦麟茜, 郑先伟. 影像解译中的深度学习可解释性分析方法[J]. 测绘学报, 2022, 51(6): 873-884.
[4]	张勤, 赵超英, 陈雪蓉. 多源遥感地质灾害早期识别技术进展与发展趋势[J]. 测绘学报, 2022, 51(6): 885-896.
[5]	龚健雅, 张觅, 胡翔云, 张展, 李彦胜, 姜良存. 智能遥感深度学习框架与模型设计[J]. 测绘学报, 2022, 51(4): 475-487.
[6]	毛文婧, 王卫林, 焦利民, 刘安宝. 基于深度学习的中国连续空间覆盖PM_2.5浓度预报[J]. 测绘学报, 2022, 51(3): 361-372.
[7]	金飞, 官恺, 刘智, 韩佳容, 芮杰, 李庆高. 无监督密集匹配特征提取网络性能分析[J]. 测绘学报, 2022, 51(3): 426-436.
[8]	何直蒙, 丁海勇, 安炳琪. 高分辨率遥感影像建筑物提取的空洞卷积E-Unet算法[J]. 测绘学报, 2022, 51(3): 457-467.
[9]	张永生, 张振超, 童晓冲, 纪松, 于英, 赖广陵. 地理空间智能研究进展和面临的若干挑战[J]. 测绘学报, 2021, 50(9): 1137-1146.
[10]	艾廷华. 深度学习赋能地图制图的若干思考[J]. 测绘学报, 2021, 50(9): 1170-1182.
[11]	龚健雅, 许越, 胡翔云, 姜良存, 张觅. 遥感影像智能解译样本库现状与研究[J]. 测绘学报, 2021, 50(8): 1013-1022.
[12]	杨必胜, 韩旭, 董震. 适用于城市场景大规模点云语义标识的深度学习网络[J]. 测绘学报, 2021, 50(8): 1059-1067.
[13]	陶超, 阴紫薇, 朱庆, 李海峰. 遥感影像智能解译:从监督学习到自监督学习[J]. 测绘学报, 2021, 50(8): 1122-1134.
[14]	晏雄锋, 艾廷华, 杨敏, 郑建滨. 地图空间形状认知的自编码器深度学习方法[J]. 测绘学报, 2021, 50(6): 757-765.
[15]	张传辉, 姚国标, 张力, 艾海滨, 满孝成, 黄鹏飞. 多分支网络联合的倾斜影像仿射不变特征匹配[J]. 测绘学报, 2021, 50(5): 641-651.