多分支网络联合的倾斜影像仿射不变特征匹配

doi:10.11947/j.AGCS.2021.20200506

测绘学报 ›› 2021, Vol. 50 ›› Issue (5): 641-651.doi: 10.11947/j.AGCS.2021.20200506

多分支网络联合的倾斜影像仿射不变特征匹配

张传辉¹, 姚国标¹, 张力², 艾海滨², 满孝成¹, 黄鹏飞¹

1. 山东建筑大学测绘地理信息学院, 山东济南 250100;
2. 中国测绘科学研究院, 北京 100830

收稿日期:2020-10-15 修回日期:2021-02-10 发布日期:2021-06-03
通讯作者: 姚国标 E-mail:yao7837005@163.com
作者简介:张传辉(1997-),男,硕士生,研究方向为深度学习影像匹配。E-mail:dplzch2020@163.com
基金资助:
国家自然科学基金（41601489）；山东省自然科学基金（ZR2015DQ007；ZR2020MD025）；山东省高等学校青创人才引育计划（0031802）

Affine invariant feature matching of oblique images based on multi-branch network

ZHANG Chuanhui¹, YAO Guobiao¹, ZHANG Li², AI Haibin², MAN Xiaocheng¹, Huang Pengfei¹

1. College of Surveying and Geo-Informatics, Shandong Jianzhu University, Jinan 250100, China;
2. Chinese Academy of Surveying and Mapping, Beijing 100830, China

Received:2020-10-15 Revised:2021-02-10 Published:2021-06-03
Supported by:
The National Natural Science Foundation of China (No. 41601489);The National Natural Science Foundation of Shandong Province (Nos. ZR2015DQ007;ZR2020MD025);The Talent Introduction Plan for Youth Innovation Team in Universities of Shandong Province (No. 0031802)

摘要/Abstract

摘要： 倾斜立体影像间存在较大仿射变形甚至透视畸变，导致现有的宽基线立体影像匹配算法容易失效或仅获得稀少匹配。基于此，本文提出一种多分支卷积网络联合的倾斜立体影像仿射不变特征匹配算法。算法首先使用Hessian算子提取初始特征邻域，接着引入三分支网络（TN）模型，并通过深度学习策略获得仿射不变特征邻域。为提高非同名相似特征的匹配效果，提出多边约束损失函数并训练多分支描述符网络（MDN）模型，继而得到具有较高区分度的描述符。然后采用最/次近邻距离比率（NNDR）测度获得同名特征，并通过随机采样一致性（RANSAC）算法剔除可能的误匹配点。最后，多组低空无人机倾斜立体影像试验结果验证了算法的有效性。

关键词: 倾斜立体影像, 卷积神经网络, 仿射不变特征, 深度学习, 影像匹配

Abstract: The available wide-baseline image matching algorithms have been prone to failure or only producing few matches, due to the complex affine deformation and perspective distortion. On this basis, we proposed a novel affine invariant feature matching algorithm for oblique stereo images based on multivariate network. In our method, we applied the Hessian algorithm to extract initial feature regions, then we constructed triplet network (TN) model, and obtained affine invariant feature regions through deep learning. To improve the matching performance of similar features, we proposed multilateral constraint loss function to train multi-branch descriptor network (MDN) model, and then generated deep learning descriptors with higher discrimination for image features. Afterwards, the conjugate features were produced by the matching metric of nearest/next distance ratio (NNDR), and eliminated possible mismatch points through random sampling consistency (RANSAC) algorithm. Finally, experiments on oblique stereo images acquired by unmanned aerial vehicle verified the effectiveness of the proposed approach.

Key words: oblique stereo images, convolutional neural network, affine invariant feature, deep learning, image matching

中图分类号:

P236

张传辉, 姚国标, 张力, 艾海滨, 满孝成, 黄鹏飞. 多分支网络联合的倾斜影像仿射不变特征匹配[J]. 测绘学报, 2021, 50(5): 641-651.

ZHANG Chuanhui, YAO Guobiao, ZHANG Li, AI Haibin, MAN Xiaocheng, Huang Pengfei. Affine invariant feature matching of oblique images based on multi-branch network[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(5): 641-651.

参考文献

[1] MIKOLAJCZYK K, SCHMID C. Scale & affine invariant interest point detectors[J]. International Journal of Computer Vision, 2004, 60(1):63-86.
[2] 姚国标, 邓喀中, 张力, 等. 基于Harris-Affine的宽基线立体影像LSM匹配方法[J]. 中南大学学报(自然科学版), 2014, 45(8):2661-2668. YAO Guobiao, DENG Kazhong, ZHANG Li, et al. Least square matching method for wide baseline stereo images based on Harris-Affine features[J]. Journal of Central South University (Science and Technology), 2014, 45(8):2661-2668.
[3] MIKOLAJCZYK K, TUYTELAARS T, SCHMID C, et al. A comparison of affine region detectors[J]. International Journal of Computer Vision, 2005, 65(1-2):43-72.
[4] MATAS J, CHUM O, URBAN M, et al. Robust wide-baseline stereo from maximally stable extremal regions[J]. Image and Vision Computing, 2004, 22(10):761-767.
[5] MOREL J M, YU Guoshen. ASIFT:a new framework for fully affine invariant image comparison[J]. SIAM Journal on Imaging Sciences, 2009, 2(2):438-469.
[6] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
[7] FAN Dazhao, DONG Yang, ZHANG Yongsheng. Satellite image matching method based on deep convolutional neural network[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(2):90-100. DOI:10.11947/j.JGGS.2019.0210.
[8] GONG Jianya, JI Shunping. Photogrammetry and deep learning[J]. Journal of Geodesy and Geoinformation Science, 2018, 1(1):1-15. DOI:10.11947/j.JGGS.2018.0101.
[9] 刘瑾, 季顺平. 基于深度学习的航空遥感影像密集匹配[J]. 测绘学报, 2019, 48(9):1141-1150. DOI:10.11947/j.AGCS.2019.20180247. LIU Jin, JI Shunping. Deep learning based dense matchingfor aerial remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(9):1141-1150. DOI:10.11947/j.AGCS.2019.20180247.
[10] 袁修孝, 袁巍, 许殊, 等. 航摄影像密集匹配的研究进展与展望[J]. 测绘学报, 2019, 48(12):1542-1550. DOI:10.11947/j.AGCS.2019.20190453. YUAN Xiuxiao, YUAN Wei, XU Shu, et al. Research developments and prospects on dense image matching in photogrammetry[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(12):1542-1550. DOI:10.11947/j.AGCS.2019.20190453.
[11] YI K M, TRULLS E, LEPETIT V, et al. LIFT:learned invariant feature transform[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands:Springer, 2016:467-483.
[12] VERDIE Y, YI K M, FUA P, et al. TILDE:a temporally invariant learned DEtector[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA:IEEE, 2015.
[13] TIAN Yurun, FAN Bin, WU Fuchao. L2-Net:deep learning of discriminative patch descriptor in Euclidean space[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI:IEEE, 2017:661-669.
[14] MISHCHUK A, MISHKIN D, RADENOVIC' F, et al. Working hard to know your neighbor's margins:local descriptor learning loss[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, CA:Curran Associates Inc., 2017:4826-4837
[15] HOFFER E, AILON N. Deep metric learning using tripletnetwork[C]//Proceedings of the 3rd International Workshop on Similarity-Based Pattern Recognition. Copenhagen, Denmark:Springer, 2015.
[16] IOFFE S, SZEGEDY C. Batch normalization:Accelerating deep network training by reducing internal covariate shift[C]//Proceedings of 2015 International Conference on Machine Learning.[S.l.]:GMLR, 2015.
[17] GLOROT X, BORDES A, BENGIO Y, et al. Deep sparse rectifier neural networks[C]//Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. Fort Lauderdale, FL:W&CP, 2011:315-323.
[18] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.[S.l.]:IEEE, 2017.
[19] YI K M, VERDIE Y, FUA P, et al. Learning to assign orientations to feature points[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016:107-116.
[20] 陈敏, 朱庆, 何海清, 等. 面向城区宽基线立体像对视角变化的结构自适应特征点匹配[J]. 测绘学报, 2019, 48(9):1129-1140. DOI:10.11947/j.AGCS.2019.20180266. CHEN Min, ZHU Qing, HE Haiqing, et al. Structureadaptive feature point matching for urban area wide-baseline images with viewpoint variation[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(9):1129-1140. DOI:10.11947/j.AGCS.2019.20180266.
[21] BROWN M, HUA Gang, WINDER S. Discriminative learning of local image descriptors[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(1):43-57.
[22] 姚国标, 邓喀中, 张力, 等. 融合互补仿射不变特征的倾斜立体影像高精度自动配准方法[J]. 测绘学报, 2013, 42(6):869-876, 883. YAO Guobiao, DENG Kazhong, ZHANG Li, et al. An automated registration method with high-accuracy for oblique stereo images based on complementary affine invariant features[J]. Acta Geodaetica et Cartographica Sinica, 2013, 42(6):869-876, 883.
[23] 李欣, 杨宇辉, 杨博, 等. 利用方向相位特征进行多源遥感影像匹配[J]. 武汉大学学报(信息科学版), 2020, 45(4):488-494. LI Xin, YANG Yuhui, YANG Bo, et al. A multi-source remote sensing image matching method using directional phase feature[J]. Geomatics and Information Science of Wuhan University, 2020, 45(4):488-494.
[24] 袁修孝, 袁巍, 陈时雨. 基于图论的遥感影像误匹配点自动探测方法[J]. 武汉大学学报(信息科学版), 2018, 43(12):1854-1860. YUAN Xiuxiao, YUAN Wei, CHEN Shiyu. An automatic detection method of mismatching points in remote sensing images based on graph theory[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12):1854-1860.
[25] TIAN Yurun, YU Xin, FAN Bin, et al. SOSNet:second order similarity regularization for local descriptor learning[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA:IEEE, 2019:11016-11025.
[26] LI S, HE F, DU B, et al. Fast spatio-temporal residual network for video super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.[S.l.]:GMLR, 2019.

多分支网络联合的倾斜影像仿射不变特征匹配

Affine invariant feature matching of oblique images based on multi-branch network

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	张良培, 何江, 杨倩倩, 肖屹, 袁强强. 数据驱动的多源遥感信息融合研究进展[J]. 测绘学报, 2022, 51(7): 1317-1337.
[2]	程涛, 张洋, James Haworth. 基于网络和图的时空智能——概念、方法和应用[J]. 测绘学报, 2022, 51(7): 1629-1639.
[3]	龚健雅, 宦麟茜, 郑先伟. 影像解译中的深度学习可解释性分析方法[J]. 测绘学报, 2022, 51(6): 873-884.
[4]	张勤, 赵超英, 陈雪蓉. 多源遥感地质灾害早期识别技术进展与发展趋势[J]. 测绘学报, 2022, 51(6): 885-896.
[5]	龚健雅, 张觅, 胡翔云, 张展, 李彦胜, 姜良存. 智能遥感深度学习框架与模型设计[J]. 测绘学报, 2022, 51(4): 475-487.
[6]	毛文婧, 王卫林, 焦利民, 刘安宝. 基于深度学习的中国连续空间覆盖PM_2.5浓度预报[J]. 测绘学报, 2022, 51(3): 361-372.
[7]	金飞, 官恺, 刘智, 韩佳容, 芮杰, 李庆高. 无监督密集匹配特征提取网络性能分析[J]. 测绘学报, 2022, 51(3): 426-436.
[8]	何直蒙, 丁海勇, 安炳琪. 高分辨率遥感影像建筑物提取的空洞卷积E-Unet算法[J]. 测绘学报, 2022, 51(3): 457-467.
[9]	张玉鑫, 颜青松, 邓非. 高分辨率遥感影像建筑物提取多路径RSU网络法[J]. 测绘学报, 2022, 51(1): 135-144.
[10]	张永生, 张振超, 童晓冲, 纪松, 于英, 赖广陵. 地理空间智能研究进展和面临的若干挑战[J]. 测绘学报, 2021, 50(9): 1137-1146.
[11]	艾廷华. 深度学习赋能地图制图的若干思考[J]. 测绘学报, 2021, 50(9): 1170-1182.
[12]	龚健雅, 许越, 胡翔云, 姜良存, 张觅. 遥感影像智能解译样本库现状与研究[J]. 测绘学报, 2021, 50(8): 1013-1022.
[13]	杨必胜, 韩旭, 董震. 适用于城市场景大规模点云语义标识的深度学习网络[J]. 测绘学报, 2021, 50(8): 1059-1067.
[14]	陶超, 阴紫薇, 朱庆, 李海峰. 遥感影像智能解译:从监督学习到自监督学习[J]. 测绘学报, 2021, 50(8): 1122-1134.
[15]	杨秋莲, 刘艳飞, 丁乐乐, 孟凡效. 高分影像场景分类的半监督深度卷积神经网络学习方法[J]. 测绘学报, 2021, 50(7): 930-938.