多源遥感影像学习型特征双向一致性配准

doi:10.11947/j.AGCS.2023.20220412

测绘学报 ›› 2023, Vol. 52 ›› Issue (11): 1906-1916.doi: 10.11947/j.AGCS.2023.20220412

多源遥感影像学习型特征双向一致性配准

张永显¹, 马国锐¹, 訾栓紧², 门行³

1. 武汉大学测绘遥感信息工程国家重点实验室, 湖北武汉 430079;
2. 中国消防救援学院, 北京 102202;
3. 65547部队, 辽宁鞍山 114200

收稿日期:2022-06-28 修回日期:2022-12-28 发布日期:2023-12-15
通讯作者: 马国锐 E-mail:mgr@whu.edu.cn
作者简介:张永显(1990-),男,博士生,研究方向为多源遥感影像自动配准。E-mail:zhyx009@whu.edu.cn
基金资助:
广西科技重大专项(AA22068072)

Multi-source remote sensing image bidirectional consistent registration based on learning feature

ZHANG Yongxian¹, MA Guorui¹, ZI Shuanjin², MEN Hang³

1. State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China;
2. China Fire and Rescue Institute, Beijing 102202, China;
3. Troops 65547, Anshan 114200, China

Received:2022-06-28 Revised:2022-12-28 Published:2023-12-15
Supported by:
The Guangxi Science and Technology Major Project (No. AA22068072)

摘要/Abstract

摘要： 针对多源遥感影像之间非线性辐射和几何畸变的差异严重影响配准质量的问题,本文提出一种具有双向一致性变换适用于多源遥感影像的配准方法。首先,利用微调的ResNet101网络模型提取多源遥感影像学习型特征,在特征匹配阶段,为提高同名特征匹配的可靠性,设计了一种双向一致性特征匹配网络模型;然后,基于小型轻量级网络加权回归变换模型参数,实现多源遥感影像稳健可靠的配准。试验利用Google Earth影像、卫星影像、无人机影像、Google Earth-卫星-无人机混合影像4种不同数据源对本文方法进行测试,并与具有代表性的多种方法进行比较,结果表明本文方法在配准精度、效率、稳健性方面具有优势,基本实现了2像素以内的自动配准精度。

关键词: 学习型特征, 特征匹配, 多源遥感影像, 影像配准, 双向一致性变换

Abstract: A robust registration method with bidirectional consistent transformation is proposed to solve the problem of multi-source remote sensing image relatively poor registration effect due to large nonlinear radiation and geometric distortion. First, the fine-tuned ResNet101 network model was utilized to extract images learning features. To improve the reliability of the corresponding feature matching, we designed a bidirectional consistency feature matching model in the feature matching stage. Then, robust registration was achieved by using a parametric regression transformation model with weight based on a small lightweight network. In our experiments, we tested the proposed algorithm using Google Earth images, satellite images, UAV images and Google Earth-satellite-UAV images, and compared it with the several typical methods. The results show that the proposed method has advantages in the accuracy, efficiency and robustness, and achieves automatic registration accuracy almost within 2 pixels.

Key words: learning feature, feature matching, multi-source images, image registration, bidirectional consistent transformation

中图分类号:

P236

张永显, 马国锐, 訾栓紧, 门行. 多源遥感影像学习型特征双向一致性配准[J]. 测绘学报, 2023, 52(11): 1906-1916.

ZHANG Yongxian, MA Guorui, ZI Shuanjin, MEN Hang. Multi-source remote sensing image bidirectional consistent registration based on learning feature[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(11): 1906-1916.

参考文献

[1] 冯蕊涛, 杜清运, 罗恒, 等. 基于光流校正的复杂地形区多时相遥感影像配准[J]. 遥感学报, 2021, 25(2):630-640. FENG Ruitao, DU Qingyun, LUO Heng, et al. A registration algorithm based on optical flow modification for multi-temporal remote sensing images covering the complex-terrain region[J]. National Remote Sensing Bulletin, 2021, 25(2):630-640.
[2] TONG Xinyi, XIA Guisong, HU Fan, et al. Exploiting deep features for remote sensing image retrieval:a systematic investigation[J]. IEEE Transactions on Big Data, 2020, 6(3):507-521.
[3] CHENG Jian, LENG Cong, WU Jiaxiang, et al. Fast and accurate image matching with cascade hashing for 3D reconstruction[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus:IEEE, 2014:1-8.
[4] BRUNNER D, LEMOINE G, BRUZZONE L. Earthquake damage assessment of buildings using VHR optical and SAR imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2010, 48(5):2403-2420.
[5] ZHANG Xu, YU F X, KARAMAN S, et al. Learning discriminative and transformation covariant local feature detectors[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu:IEEE, 2017:4923-4931.
[6] FAN Bin, HUO Chunlei, PAN Chunhong, et al. Registration of optical and SAR satellite images by exploring the spatial relationship of the improved SIFT[J]. IEEE Geoscience and Remote Sensing Letters, 2013, 10(4):657-661.
[7] XIANG Yuming, WANG Feng, YOU Hongjian. OS-SIFT:a robust SIFT-like algorithm for high-resolution optical-to-SAR image registration in suburban areas[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(6):3078-3090.
[8] YU Qiuze, NI Dawen, JIANG Yuxuan, et al. Universal SAR and optical image registration via a novel SIFT framework based on nonlinear diffusion and a polar spatial-frequency descriptor[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 171:1-17.
[9] ZHANG Wannan. Combination of SIFT and canny edge detection for registration between SAR and optical images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19:1-5.
[10] 李培, 姜刚, 马千里, 等. 结合张量与互信息的混合模型多模态图像配准方法[J]. 测绘学报, 2021, 50(7):916-929. DOI:10.11947/j.AGCS.2021.20200492. LI Pei, JIANG Gang, MA Qianli, et al. A hybrid model combining tensor and mutual information for multi-modal image registration[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(7):916-929. DOI:10.11947/j.AGCS.2021.20200492.
[11] 叶沅鑫, 单杰, 彭剑威, 等. 利用局部自相似进行多光谱遥感图像自动配准[J]. 测绘学报, 2014, 43(3):268-275. YE Yuanxin, SHAN Jie, PENG Jianwei, et al. Automated multispectral remote sensing image registration using local self-similarity[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(3):268-275.
[12] YE Yuanxin, BRUZZONE L, SHAN Jie, et al. Fast and robust matching for multimodal remote sensing image registration[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(11):9059-9070.
[13] YE Yuanxin, SHAN Jie, BRUZZONE L, et al. Robust registration of multimodal remote sensing images based on structural similarity[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(5):2941-2958.
[14] 张力, 刘玉轩, 孙洋杰, 等. 数字航空摄影三维重建理论与技术发展综述[J]. 测绘学报, 2022, 51(7):1437-1457. DOI:10.11947/j.AGCS.2022.20220130. ZHANG Li, LIU Yuxuan, SUN Yangjie, et al. A review of developments in the theory and technology of three-dimensional reconstruction in digital aerial photogrammetry[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(7):1437-1457. DOI:10.11947/j.AGCS.2022.20220130.
[15] 蓝朝桢, 卢万杰, 于君明, 等. 异源遥感影像特征匹配的深度学习算法[J]. 测绘学报, 2021, 50(2):189-202. DOI:10.11947/j.AGCS.2021.20200048. LAN Chaozhen, LU Wanjie, YU Junming, et al. Deep learning algorithm for feature matching of cross modality remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(2):189-202. DOI:10.11947/j.AGCS.2021.20200048.
[16] FAN Rongbo, HOU Bochuan, LIU Jinbao, et al. Registration of multiresolution remote sensing images based on L2-siamese model[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 14:237-248.
[17] ZHANG Han, NI Weiping, YAN Weidong, et al. Registration of multimodal remote sensing image based on deep fully convolutional neural network[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(8):3028-3042.
[18] BÜRGMANN T, KOPPE W, SCHMITT M. Matching of TerraSAR-X derived ground control points to optical image patches using deep learning[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 158:241-248.
[19] DUSMANU M, ROCCO I, PAJDLA T, et al. D2-net:a trainable CNN for joint description and detection of local features[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach:IEEE, 2020:8084-8093.
[20] SARLIN P E, DETONE D, MALISIEWICZ T, et al. SuperGlue:learning feature matching with graph neural networks[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle:IEEE, 2020:4938-4947.
[21] CHEN Hongkai, LUO Zixin, ZHANG Jiahui, et al. Learning to match features with seeded graph matching network[C]//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal:IEEE, 2022:6281-6290.
[22] WANG Shuang, QUAN Dou, LIANG Xuefeng, et al. A deep learning framework for remote sensing image registration[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 145:148-164.
[23] HUGHES L H, MARCOS D, LOBRY S, et al. A deep learning framework for matching of SAR and optical imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 169:166-179.
[24] 朱庆涛, 殷君君, 曾亮, 等. 基于邻域一致性的极化SAR图像仿射配准[J]. 雷达学报, 2021, 10(1):49-60. ZHU Qingtao, YIN Junjun, ZENG Liang, et al. Polarimetric SAR image affine registration based on neighborhood consensus[J]. Journal of Radars, 2021, 10(1):49-60.
[25] ROCCO I, ARANDJELOVIC' R, SIVIC J. Convolutional neural network architecture for geometric matching[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(11):2553-2567.
[26] PARK J H, NAM W J, LEE S W. A two-stream symmetric network with bidirectional ensemble for aerial image matching[J]. Remote Sensing, 2020, 12(3):465.
[27] LI Jiayuan, HU Qingwu, AI Mingyao. RIFT:multi-modal image matching based on radiation-variation insensitive feature transform[J]. IEEE Transactions on Image Processing, 2020, 29:3296-3310.

多源遥感影像学习型特征双向一致性配准

Multi-source remote sensing image bidirectional consistent registration based on learning feature

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 12

编辑推荐

Metrics

本文评价

[1]	宋佳璇, 范大昭, 董杨, 纪松, 李东子. 神经网络学习与灰度信息结合的跨视角影像线特征匹配算法[J]. 测绘学报, 2023, 52(6): 990-999.
[2]	张广运, 韩一, 张荣庭, 李明峰, 吉文来. 基于空间一致性的同平台点云配准方法[J]. 测绘学报, 2023, 52(5): 789-797.
[3]	龚健雅, 许越, 胡翔云, 姜良存, 张觅. 遥感影像智能解译样本库现状与研究[J]. 测绘学报, 2021, 50(8): 1013-1022.
[4]	刘嗣超, 武鹏达, 赵占杰, 李成名. 交通监控视频图像语义分割及其拼接方法[J]. 测绘学报, 2020, 49(4): 522-532.
[5]	欧阳欢, 范大昭, 纪松, 雷蓉. 结合离散化描述与同名点约束的线特征匹配[J]. 测绘学报, 2018, 47(10): 1363-1371.
[6]	黄波, 赵涌泉. 多源卫星遥感影像时空融合研究的现状及展望[J]. 测绘学报, 2017, 46(10): 1492-1499.
[7]	王慧贤, 靳惠佳, 雷呈强, 江万寿, 王艳. 视觉驱动的变分配准方法[J]. 测绘学报, 2015, 44(8): 893-899.
[8]	陈驰, 杨必胜, 彭向阳. 低空UAV激光点云和序列影像的自动配准方法[J]. 测绘学报, 2015, 44(5): 518-525.
[9]	陈强, 罗容, 杨莹辉, 雍琦. 利用SAR影像配准偏移量提取地表形变的方法与误差分析[J]. 测绘学报, 2015, 44(3): 301-308.
[10]	逯跃锋. 基于形状特征的矢量数据与影像数据配准方法研究[J]. 测绘学报, 2014, 43(8): 879-879.
[11]	罗楠孙权森耿蕾蕾李辉夏德深. 一种扩展SURF描述符及其在遥感图像配准中的应用[J]. , 2013, 42(3): 0-0.
[12]	王晏民,胡春梅. 一种地面激光雷达点云与纹理影像稳健配准方法[J]. 测绘学报, 2012, 41(2): 266-272.