结合张量与互信息的混合模型多模态图像配准方法

doi:10.11947/j.AGCS.2021.20200492

测绘学报 ›› 2021, Vol. 50 ›› Issue (7): 916-929.doi: 10.11947/j.AGCS.2021.20200492

结合张量与互信息的混合模型多模态图像配准方法

李培¹, 姜刚^1,2, 马千里¹, 薛万峰¹, 杨伟华¹

1. 长安大学地质与测绘工程学院, 陕西西安 710054;
2. 西部矿产资源与地质工程教育部重点实验室, 陕西西安 710054

收稿日期:2020-10-21 修回日期:2021-04-21 发布日期:2021-08-13
作者简介:李培(1996-),男,硕士,研究方向为多模态图像配准。E-mail:1239608938@qq.com
基金资助:
国家自然科学基金（41977231）

A hybrid model combining tensor and mutual information for multi-modal image registration

LI Pei¹, JIANG Gang^1,2, MA Qianli¹, XUE Wanfeng¹, YANG Weihua¹

1. College of Geology Engineering and Geomatics, Chang'an University, Xi'an 710054, China;
2. Key Laboratory of Western China's Mineral Resources and Geological Engineering, Ministry of Education, Xi'an 710054, China

Received:2020-10-21 Revised:2021-04-21 Published:2021-08-13
Supported by:
The National Natural Science Foundation of China (No. 41977231)

摘要/Abstract

摘要： 多模态图像之间存在显著的非线性强度差异，并且图像会因为噪声而退化，因此，多模态图像自动配准是一项具有挑战性的任务。为了解决这两个问题，本文提出一种多模态图像自动配准方法，该方法分为预配准和精配准两个阶段。在预配准阶段，通过改进SIFT算法来大致对齐多模态图像。在精配准阶段，首先，利用块Harris检测器在预配准后的参考图上提取均匀分布的特征点。然后，通过各向异性结构张量捕捉多模态图像中的结构信息来构建特征描述符，该特征描述符对噪声具有稳健性。更进一步，本文结合张量方向平行度和梯度互信息提出了一种相似度准则（tensor orientation and mutual information，TOMI）。最后，本文用多种模态图像（包括Optical，LiDAR，SAR和Map）来评估提出的方法。试验结果表明，本文提出的方法对非线性强度变化和噪声具有较好的稳健性，并且匹配效果优越。

关键词: 各向异性滤波, 多模态图像, 结构张量, 相似度准则, 图像配准

Abstract: There are significant nonlinear intensity differences between multi-modal images. Moreover, the noise in these images will cause image degradation. Therefore, the automatic registration of multi-modal images is a challenging task. To address the two problems, this paper proposes a multi-modal image automatic registration method, which is divided into two stages: pre-registration and fine registration. In the pre-registration stage, an improved SIFT algorithm is used to roughly align multi-modal images. In the fine registration stage, the block Harris detector is first used to extract evenly distributed feature points on the pre-registered reference image. Then, the structure information in the multi-modal images is captured by the anisotropic structure tensor to construct a feature descriptor, which is robust to noise. Furthermore, a similarity criterion named TOMI (tensor orientation and mutual information) is proposed combining the tensor orientation parallelism and gradient mutual information. Finally, Multi-modal images (including Optical, LiDAR, SAR, and Map data) are used to evaluate the proposed algorithm. The experimental results show that the method proposed in this paper is robust to nonlinear intensity differences and noise, and the matching effect is superior.

Key words: anisotropic filtering, multi-modal images, structure tensor, similarity criterion, image registration

中图分类号:

P237

李培, 姜刚, 马千里, 薛万峰, 杨伟华. 结合张量与互信息的混合模型多模态图像配准方法[J]. 测绘学报, 2021, 50(7): 916-929.

LI Pei, JIANG Gang, MA Qianli, XUE Wanfeng, YANG Weihua. A hybrid model combining tensor and mutual information for multi-modal image registration[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(7): 916-929.

参考文献

[1] YE Yuanxin, SHEN Li. HOPC: a novel similarity metric based on geometric structural properties for multi-modal remote sensing image matching[J]. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016, Ⅲ-1: 9-16.
[2] YE Yuanxin, SHAN Jie, BRUZZONE L, et al. Robust registration of multimodal remote sensing images based on structural similarity[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(5): 2941-2958.
[3] LI Jiayuan, HU Qingwu, AI Mingyao. Optimal illumination and color consistency for optical remote-sensing image mosaicking[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(11): 1943-1947.
[4] LIU Jianchen, GONG Jianya, GUO Bingxuan, et al. A novel adjustment model for mosaicking low-overlap sweeping images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7): 4089-4097.
[5] LAZEBNIK S, SCHMID C, PONCE J. Beyond bags of features: spatial pyramid matching for recognizing natural scene categories[C]//Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). New York, NY, USA: IEEE, 2006: 2169-2178.
[6] 冯文卿, 张永军. 利用多尺度融合进行面向对象的遥感影像变化检测[J]. 测绘学报, 2015, 44(10): 1142-1151. DOI: 10.11947/j.AGCS.2015.20140260. FENG Wenqing, ZHANG Yongjun. Object-oriented change detection for remote sensing images based on multiscale fusion[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(10): 1142-1151. DOI: 10.11947/j.AGCS.2015.20140260.
[7] 张良培, 武辰. 多时相遥感影像变化检测的现状与展望[J]. 测绘学报, 2017, 46(10): 1447-1459.DOI: 10.11947/j.AGCS.2017.20170340. ZHANG Liangpei, WU Chen. Advance and future development of change detection for multi-temporal remote sensing imagery[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10): 1447-1459.DOI: 10.11947/j.AGCS.2017.20170340.
[8] MA Wenping, WEN Zelian, WU Yue, et al. Remote sensing image registration with modified SIFT and enhanced feature matching[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(1): 3-7.
[9] ZITOVÁ B, FLUSSER J. Image registration methods: a survey[J]. Image and Vision Computing, 2003, 21(11): 977-1000.
[10] YE Yuanxin, SHAN Jie. A local descriptor based registration method for multispectral remote sensing images with non-linear intensity differences[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014, 90: 83-95.
[11] YE Yuanxin, BRUZZONE L, SHAN Jie, et al. Fast and robust matching for multimodal remote sensing image registration[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(11): 9059-9070.
[12] XIANG Yuming, WANG Feng, YOU Hongjian. OS-SIFT: a robust SIFT-like algorithm for high-resolution optical-to-SAR image registration in suburban areas[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(6): 3078-3090.
[13] LI Jiayuan, HU Qingwu, AI Mingyao. RIFT: multi-modal image matching based on radiation-variation insensitive feature transform[J]. IEEE Transactions on Image Processing, 2020, 29: 3296-3310.
[14] KOVESI P. Image features from phase congruency[J]. Videre: Journal of Computer Vision Research, 1999, 1(3): 1-26.
[15] COLE-RHODES A A, JOHNSON K L, LEMOIGNE J, et al. Multiresolution registration of remote sensing imagery by optimization of mutual information using a stochastic gradient[J]. IEEE Transactions on Image Processing, 2003, 12(12): 1495-1511.
[16] PLUIM J P W, MAINTZ J B A, VIERGEVER M A. Image registration by maximization of combined mutual information and gradient information[C]//Proceedings of 2000 International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Pittsburgh, PA, USA: Springer, 2000: 452-461.
[17] SURI S, REINARTZ P. Mutual-information-based registration of TerraSAR-X and IKONOS imagery in urban areas[J]. IEEE Transactions on Geoscience and Remote Sensing, 2010, 48(2): 939-949.
[18] LU Jiazhen, HU Maoqing, DONG Jing, et al. A novel dense descriptor based on structure tensor voting for multi-modal image matching[J]. Chinese Journal of Aeronautics, 2020, 33(9): 2408-2419.
[19] LOWE D G. Distinctive image features from scale: invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
[20] YU Le, ZHANG Dengrong, HOLDEN E J. A fast and fully automatic registration approach based on point features for multi-source remote-sensing images[J]. Computers & Geosciences, 2008, 34(7): 838-848.
[21] DELLINGER F, DELON J, GOUSSEAU Y, et al. SAR-SIFT: a SIFT-like algorithm for SAR images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(1): 453-466.
[22] MIKOLAJCZYK K, SCHMID C. Indexing based on scale invariant interest points[C]//Proceedings of the 8th IEEE International Conference on Computer Vision.Vancouver, BC, Canada:IEEE, 2001: 525-531.
[23] BIGVN J, GRANLUND G H, WIKLUND J. Multidimen-sional orientation estimation with applications to texture analysis and optical flow[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1991, 13(8): 775-790.
[24] BROX T, WEICKERT J, BURGETH B, et al. Nonlinear structure tensors[J]. Image and Vision Computing, 2006, 24(1): 41-55.
[25] KÖTHE U. Edge and junction detection with an improved structure tensor[M]//Lecture Notes in Computer Science. Berlin: Springer, 2003: 25-32.
[26] YANG G Z, BURGER P, FIRMIN D N, et al. Structure adaptive anisotropic image filtering[J]. Image and Vision Computing, 1996, 14(2): 135-145.

结合张量与互信息的混合模型多模态图像配准方法

A hybrid model combining tensor and mutual information for multi-modal image registration

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 5

编辑推荐

Metrics

本文评价

[1]	程争刚, 张利. 一种基于无人机位姿信息的航拍图像拼接方法[J]. 测绘学报, 2016, 45(6): 698-705.
[2]	王青松,瞿继双,黄海风,余安喜,董臻. 联合实、复相关函数的干涉SAR图像配准方法[J]. 测绘学报, 2012, 41(4): 0-590.
[3]	杨化超,张磊,姚国标,王永波. 局部单应约束的高精度图像自动配准方法[J]. 测绘学报, 2012, 41(3): 0-416.
[4]	钟成. 集成LiDAR和遥感影像生成高质量真正射影像研究[J]. 测绘学报, 2011, 40(1): 132-132.
[5]	卜彦龙潘亮彭辉沈林成. 未知DEM下的机载SAR图像几何校正方法研究[J]. 测绘学报, 2009, 38(1): 0-11.