A fast matching method of SAR and optical images using angular weighted orientated gradients

doi:10.11947/j.AGCS.2021.20200587

Abstract

Abstract: To solve the problem of matching difficulty caused by the significant nonlinear grayscale differences between SAR and optical images, this paper proposes a fast matching algorithm based on image structural properties named SOFM(SAR-to-optical fast matching algorithm).The traditional methods based on image grayscale are generally difficult to resist the nonlinear grayscale differences between SAR and optical images, but the geometric constructs and shape features can exist stably among different types of images, so in our the proposed method both the magnitude and orientation information of image gradient are used to build a geometric structural feature descriptor named AWOG(angular weighted orientated gradients), then based on the template matching strategy, the sum of squared difference of the descriptors is used to define the similarity metric for matching and then the image matching function expressed in the frequency domain is given. A complete set of image matching process is established based on SOFM, and has been validated using multiple pairs of SAR and optical images, the results show that the proposed method can effectively resist the nonlinear grayscale differences between SAR and optical images, and outperforms the traditional classical image grayscale-based methods and existing image structural-based methods in matching performance and precision.

Key words: SAR images, optical images, structural properties, image gradient, template matching

CLC Number:

P237

FAN Zhongli, ZHANG Li, WANG Qingdong, LIU Siting, YE Yuanxin. A fast matching method of SAR and optical images using angular weighted orientated gradients[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(10): 1390-1403.

References

[1] WANG Yafei, ZHAO Feifei, CHENG Liang, et al. Framework for monitoring the conversion of cultivated land to construction land using SAR image time series[J]. Remote Sensing Letters, 2015, 6(10):794-803.
[2] STROZZI T, U WEGMVLLER, TOSI L, et al. Land subsidence monitoring with differential SAR interferometry[J]. Photogrammetric Engineering & Remote Sensing, 2001, 67(11):1261-1270.
[3] 叶沅鑫, 慎利, 陈敏, 等. 局部相位特征描述的多源遥感影像自动匹配[J]. 武汉大学学报(信息科学版), 2017, 42(9):1278-1284. YE Yuanxin, SHEN Li, CHEN Min, et al. An automatic matching method based on local phase feature descriptor for multi-source remote sensing images[J]. Geomatics and Information Science of Wuhan University, 2017, 42(9):1278-1284.
[4] 陈敏, 朱庆, 朱军, 等. SAR影像与光学影像的高斯伽玛型边缘强度特征匹配法[J]. 测绘学报, 2016, 45(3):318-325. CHEN Min, ZHU Qing, ZHU Jun, et al. Feature matching for SAR and optical images based on Gaussian-gammashaped edge strength map[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(3):318-325.
[5] 闫利, 王紫琦, 叶志云. 顾及灰度和梯度信息的多模态影像配准算法[J]. 测绘学报, 2018, 47(1):71-81. YAN Li, WANG Ziqi, YE Zhiyun. Multimodal image registration algorithm considering grayscale and gradient information[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(1):71-81.
[6] 岳春宇, 江万寿. 几何约束和改进SIFT的SAR影像和光学影像自动配准方法[J]. 测绘学报, 2012, 41(4):570-576. YUE Chunyu, JIANG Wanshou. An automatic registration algorithm for SAR and optical images based on geometry constraint and improved SIFT[J]. Acta Geodaetica et Cartographica Sinica, 2012, 41(4):570-576.
[7] 李雨谦, 皮亦鸣, 王金峰. 基于水平集的SAR图像与光学图像的配准[J]. 测绘学报, 2010, 39(3):276-282. LI Yuqian, PI Yiming, WANG Jinfeng. The registration between SAR and optical image based on level set[J]. Acta Geodaetica et Cartographica Sinica, 2010, 39(3):276-282.
[8] ZITOVÁ B, FLUSSER J. Image registration methods:a survey[J]. Image and Vision Computing, 2003, 21(11):977-1000.
[9] ZHANG Jun, MA Wenping, WU Yue, et al. Multimodal remote sensing image registration based on image transfer and local features[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(8):1210-1214.
[10] 南轲, 齐华, 叶沅鑫. 深度卷积特征表达的多模态遥感影像模板匹配方法[J]. 测绘学报, 2019, 48(6):727-736. NAN Ke, QI Hua, YE Yuanxin. A template matching method of multimodal remote sensing images based on deep convolutional feature representation[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6):727-736.
[11] YU Le, ZHANG Dengrong, HOLDEN E J. A fast and fully automatic registration approach based on point features for multi-source remote-sensing images[J]. Computers & Geosciences, 2008, 34(7):838-848.
[12] CHEN Min, SHAO Zhenfeng. Robust affine-invariant line matching for high resolution remote sensing images[J]. Photogrammetric Engineering & Remote Sensing, 2013, 79(8):753-760.
[13] GONCALVES H, CORTE-REAL L, GONCALVES J A. Automatic image registration through image segmentation and SIFT[J]. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(7):2589-2600.
[14] LOWE D G. Object recognition from local scale-invariant features[C]//Proceedings of the 7th IEEE International Conference on Computer Vision. Kerkyra, Greece:IEEE, 1999:1150-1157.
[15] BAY H, TUYTELAARS T, VAN GOOL L. SURF:speeded up robust features[C]//Proceedings of the European conference on computer vision. Berlin, Heidelberg:Springer, 2006:404-417.
[16] 叶沅鑫, 郝思媛, 曹云刚. 基于几何结构属性的光学和SAR影像自动配准[J]. 红外与毫米波学报, 2017, 36(6):720-726. YE Yuanxin, HAO Siyuan, CAO Yungang. Automatic registration of optical and SAR image using geometric structural properties[J]. Journal of Infrared and Millimeter Waves, 2017, 36(6):720-726.
[17] COLE-RHODES A A, JOHNSON K L, LEMOIGNE J, et al. Multiresolution registration of remote sensing imagery by optimization of mutual information using a stochastic gradient[J]. IEEE Transactions on Image Processing, 2003, 12(12):1495-1511.
[18] MARTINEZ A, GARCIA-CONSUEGRA J, ABAD F. A correlation-symbolic approach to automatic remotely sensed image rectification[C]//Proceedings of 1999 IEEE International Geoscience and Remote Sensing Symposium. Hamburg, Germany:IEEE, 1999:336-338.
[19] BROMLEY J, BENTZ J W, BOTTOU L, et al. Signature verification using a "Siamese" time delay neural network[J]. Advances in neural information processing systems, 1994(1):737-737.
[20] BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional siamese networks for object tracking[C]//Proceedings of the European Conference on Computer Vision. Cham:Springer, 2016.
[21] SHECHTMAN E, IRANI M. Matching local self-similarities across images and videos[C]//Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition.Minneapolis, MN, USA:IEEE, 2007:1-8.
[22] 王峰, 尤红建. 基于级联变换的光学和SAR图像配准算法[J]. 红外与毫米波学报, 2015, 34(4):486-492. WANG Feng, YOU Hongjian. Robust registration method of SAR and optical remote sensing Images based on cascade transforms[J]. Journal of Infrared and Millimeter Waves, 2015, 34(4):486-492.
[23] YE Yuanxin, SHEN Li. Hopc:a novel similarity metric based on geometric structural properties for multi-modal remote sensing image matching[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016, III-1:9-16.
[24] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, CA, USA:IEEE, 2005:886-893.
[25] YE Yuanxin, BRUZZONE L, SHAN Jie, et al. Fast and robust matching for multimodal remote sensing image registration[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(11):9059-9070.
[26] FELZENSZWALB P F, GIRSHICK R B, MCALLESTER D, et al. Object detection with discriminatively trained part-based models[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(9):1627-1645.
[27] HARZALLAH H, JURIE F, SCHMID C. Combining efficient object localization and image classification[C]//Proceedings of the 12th IEEE International Conference on Computer Vision. Kyoto, Japan:IEEE, 2009:237-244.
[28] SHRIVASTAVA A, MALISIEWICZ T, GUPTA A, et al. Data-driven visual similarity for cross-domain image matching[C]//Proceedings of the 2011 SIGGRAPH Asia Conference.Hong Kong, China:ACM Press, 2011:1-10.
[29] SEDAGHAT A, EBADI H. Distinctive order based self-similarity descriptor for multi-sensor remote sensing image matching[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 108:62-71.
[30] LU Jiazhen, SUN Fangde, DONG Jing. A novel multi-sensor image matching algorithm based on adaptive multiscale structure orientation[J]. IEEE Access, 2019, 7:177474-177483.
[31] REDDY B S, CHATTERJI B N. An FFT-based technique for translation, rotation, and scale-invariant image registration[J]. IEEE Trans Image Process, 1996, 5(8):1266-1271.
[32] YE Yuanxin, BRUZZONE L, SHAN Jie, et al. Fast and robust structure-based multimodal geospatial image matching[C]//Proceedings of 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Fort Worth, TX, USA:IEEE, 2017:5141-5144.
[33] YE Yuanxin, SHEN Li, HAO Ming, et al. Robust optical-to-SAR image matching based on shape properties[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(4):564-568.
[34] 楼良盛, 刘志铭, 张昊, 等. 天绘二号卫星工程设计与实现[J]. 测绘学报, 2020, 49(10):1252-1264. LOU Liangsheng, LIU Zhiming, ZHANG Hao, et al. TH-2 satellite engineering design and implementation[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(10):1252-1264.