异源遥感影像特征匹配的深度学习算法

doi:10.11947/j.AGCS.2021.20200048

摘要/Abstract

摘要： 针对异源遥感影像的成像模式、时相、分辨率等不同导致匹配困难的问题，提出了一种基于深度学习特征的匹配方法CMM-Net。首先，利用卷积神经网络提取异源遥感影像的高维特征图，根据同时满足通道最大和局部最大两种条件选取关键点，并在特征图上提取相应位置的512维描述符。在匹配阶段，完成快速最近邻搜索特征匹配后，为解决误匹配点多的问题，提出了动态自适应欧氏距离阈值和RANSAC共同约束的提纯算法，保证误匹配有效剔除的同时，最大限度保留正确匹配点。利用多组异源遥感影像对算法进行了测试，并与多种异源影像匹配算法进行了比较，结果表明本文算法能够提取出异源影像的尺度不变相似特征，具有较强的适应性和稳健性。

关键词: 深度学习, 影像匹配, 异源影像, 卷积神经网络, 卫星影像

Abstract: Focusing on the problem of difficulty in matching due to the differences in imaging modality, time phases, and resolutions of cross modality remote sensing images, a new deep learning feature matching method named CMM-Net is proposed. First, a convolutional neural network is used to extract high-dimensional feature maps of the cross modality remote sensing images. The key points are selected according to the conditions that both the channel maximum and local maximum are met, and the 512-dimensional descriptors in corresponding location are extracted on the feature map to complete the feature extraction. In the matching stage, after completing the fast-nearest neighbor searching, in order to solve the problem of lots of mismatched points, a purification algorithm with dynamic adaptive Euclidean distance and RANSAC constraints is proposed to ensure that the mismatches are effectively eliminated while retaining the correct matches. The algorithm was tested using multiple sets of cross modality remote sensing images and compared with other algorithms. The results show that the proposed algorithm has the ability to extract similar scale invariant features in cross modality images, and has strong adaptability and robustness.

Key words: deep learning, image matching, cross modality image, convolution neural network, satellite image

中图分类号:

P236

蓝朝桢, 卢万杰, 于君明, 徐青. 异源遥感影像特征匹配的深度学习算法[J]. 测绘学报, 2021, 50(2): 189-202.

LAN Chaozhen, LU Wanjie, YU Junming, XU Qing. Deep learning algorithm for feature matching of cross modality remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(2): 189-202.

参考文献

[1] ZHAO Y, JIANG M, MA Z F. Integration of SAR polarimetric features and multi-spectral data for object-based land cover classification[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(4):64-72.
[2] 李德仁. 展望大数据时代的地球空间信息学[J]. 测绘学报, 2016, 45(4):379-384. DOI:10.11947/j.AGCS.2016. 20160057. LI Deren. Towards geo-spatial information science in big data era[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(4):379-384. DOI:10.11947/j.AGCS.2016.20160057.
[3] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
[4] 戴激光, 宋伟东, 李玉. 渐进式异源光学卫星影像SIFT匹配方法[J]. 测绘学报, 2014, 43(7):746-752. DOI:10.13485/j.cnki.11-2089.2014.00. DAI Jiguang, SONG Weidong, LI Yu. Progressive SIFT matching algorithm for multi-source optical satellite images[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(7):746-752. DOI:10.13485/j.cnki.11-2089.2014.00.
[5] 王峰, 尤红建, 傅兴玉, 等. 应用于多源SAR图像匹配的级联SIFT算法[J]. 电子学报, 2016, 44(3):548-554. WANG Feng, YOU Hongjian, FU Xingyu, et al. Cascade SIFT matching method for multi-source SAR images[J]. Acta Electronica Sinica, 2016, 44(3):548-554.
[6] 叶沅鑫, 单杰, 熊金鑫, 等. 一种结合SIFT和边缘信息的多源遥感影像匹配方法[J]. 武汉大学学报(信息科学版), 2013, 38(10):1148-1151, 1260. YE Yuanxin, SHAN Jie, XIONG Jinxin, et al. A node localization method in wireless sensor network based on k-means cluster[J]. Geomatics and Information Science of Wuhan University, 2013, 38(10):1148-1151, 1260.
[7] 王瑞瑞, 马建文, 陈雪. 多传感器影像配准中基于虚拟匹配窗口的SIFT算法[J]. 武汉大学学报(信息科学版), 2011, 36(2):163-166. WANG Ruirui, MA Jianwen, CHEN Xue. SIFT algorithm based on visual matching window for registration between multi-sensor imagery[J]. Geomatics and Information Science of Wuhan University, 2011, 36(2):163-166.
[8] DELLINGER F, DELON Julie, GOUSSEAU Y, et al. SAR-SIFT:a SIFT-like algorithm for SAR images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(1):453-466.
[9] 凌志刚, 梁彦, 程咏梅, 等. 一种稳健的多源遥感图像特征配准方法[J]. 电子学报, 2010, 38(12):2892-2897. LING Zhigang, LIANG Yan, CHENG Yongmei, et al. A robust multi-source remote-sensing image registration method based on feature matching[J]. Acta Electronica Sinica, 2010, 38(12):2892-2897.
[10] KOVESI P. Image features from phase congruency[J]. Journal of Computer Vision Research, 1999, 1(3):1-26.
[11] WONG A, CLAUSI D A. ARRSI:automatic registration of remote-sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2007, 45(5):1483-1493.
[12] 陈敏, 朱庆, 朱军, 等. 多光谱遥感影像亮度空间相位一致性特征点检测[J]. 测绘学报, 2016, 45(2):178-185. DOI:10.11947/j.AGCS.2016.20150030. CHEN Min, ZHU Qing, ZHU Jun, et al. Interest point detection for multispectral remote sensing image using phase congruency in illumination space[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(2):178-185. DOI:10.11947/j.AGCS.2016.20150030.
[13] 叶沅鑫, 单杰, 彭剑威, 等. 利用局部自相似进行多光谱遥感图像自动配准[J]. 测绘学报, 2014, 43(3):268-275. DOI:10.13485/j.cnki.11-2089.2014.0039. YE Yuanxin, SHAN Jie, PENG Jianwei, et al. Automated multispectral remote sensing image registration using local self-similarity[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(3):268-275. DOI:10.13485/j.cnki.11-2089.2014.0039.
[14] 闫利, 王紫琦, 叶志云. 顾及灰度和梯度信息的多模态影像配准算法[J]. 测绘学报, 2018, 47(1):71-81. DOI:10.11947/j.AGCS.2018.20170368. YAN Li, WANG Ziqi, YE Zhiyun. Multimodal image registration algorithm considering grayscale and gradient information[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(1):71-81. DOI:10.11947/j.AGCS.2018.20170368.
[15] LI Jiayuan, HU Qingwu, AI Mingyao. RIFT:multi-modal image matching based on radiation-variation insensitive feature transform[J]. IEEE Transactions on Image Processing, 2020, 29:3296-3310.
[16] DOSOVITSKIY A, FISCHER P, SPRINGENBERG J T, et al. Discriminative unsupervised feature learning with exemplar convolutional neural networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(9):1734-1747.
[17] 龚健雅, 季顺平. 摄影测量与深度学习[J]. 测绘学报, 2018, 47(6):693-704. DOI:10.11947/j.AGCS.2018.20170640. GONG Jianya, JI Shunping. Photogrammetry and deep learning[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6):693-704. DOI:10.11947/j.AGCS.2018.20170640.
[18] ZAGORUYKO S, KOMODAKIS N. Learning to compare image patches via convolutional neural networks[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA:IEEE, 2015:4353-4361.
[19] 范大昭, 董杨, 张永生. 卫星影像匹配的深度卷积神经网络方法[J]. 测绘学报, 2018, 47(6):844-853. DOI:10.11947/j.AGCS.2018.20170627. FAN Dazhao, DONG Yang, ZHANG Yongsheng. Satellite image matching method based on deep convolution neural network[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6):844-853. DOI:10.11947/j.AGCS.2018.20170627.
[20] YANG Zhuoqian, DAN Tingting, YANG Yang. Multi-temporal remote sensing image registration using deep convolutional features[J]. IEEE Access, 2018, 6:38544-38555.
[21] YE Famao, SU Yanfei, XIAO Hui, et al. Remote sensing image registration using convolutional neural network features[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(2):232-236.
[22] GONG J Y, JI S P. Photogrammetry and deep learning[J]. Journal of Geodesy and Geoinformation Science, 2018, 1(1):1-15.
[23] YI K M, TRULLS E, LEPETIT V, et al. LIFT:learned invariant feature transform[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, Netherlands:Springer, 2016:467-483.
[24] DETONE D, MALISIEWICZ T, RABINOVICH A. SuperPoint:self-supervised interest point detection and description[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Salt Lake City, UT:IEEE, 2018:224-236.
[25] NOH H, ARAUJO A, SIM J, et al. Large-scale image retrieval with attentive deep local features[C]//2017 IEEE International Conference on Computer Vision (ICCV). Venice:IEEE, 2017:3456-3465.
[26] DUSMANU M, ROCCO I, PAJDLA T, et al. D2-Net:a trainable CNN for joint description and detection of local features[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA:IEEE, 2019.
[27] LUO Zixin, SHEN Tianwei, ZHOU Lei, et al. ContextDesc:local descriptor augmentation with cross-modality context[C]//The Processdings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA:IEEE, 2019:2522-2531.
[28] KAREN Simonyan, REW Zisserman. Very deep convolutional networks for large-scale image recognition[C]//Proceedings of the International Conference on Learning Representation. San-diego, California, USA:[s.n.],2014.
[29] MISHCHUK A, MISHKIN D, RADENOVIĆ F, et al. Working hard to know your neighbor's margins:local descriptor learning loss[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach:Curran Associates Inc., 2017.
[30] LI Zhengqi, SNAVELY N. MegaDepth:learning single-view depth prediction from internet photos[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT:IEEE, 2018.
[31] SCHÖNBERGER J L, FRAHM J M. Structure-from-motion revisited[C]//The Processdings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016:4104-4113.