Structureadaptive feature point matching for urban area wide-baseline images with viewpoint variation

doi:10.11947/j.AGCS.2019.20180266

Abstract

Abstract: A feature point matching method based on structureadaptive feature is proposed for urban area wide-baseline images. Firstly, interest points and straight lines are detected from images. Structure adaptive feature region and descriptor are constructed by exploring the geometric relationship between the interest point and the straight lines located in the local neighborhood of the point, and initial matching results are obtained by using bi-directional matching strategy. Secondly, the fundamental matrix is estimated from the initial matching results. An epipolar geometry constrained structure adaptive feature matching method is proposed to match those features have not been matched in the initial matching step. Finally, a matching expansion method is proposed based on the previous matching results to improve the matching performance.The proposed matching method can generate similar feature regions and descriptors for corresponding features under significant image viewpoint variation benefiting from the proposed structure adaptive feature region construction method. The experimental results demonstrate that the proposed method provides significant improvements in correct matches number and matching precision compared with other traditional matching methods for urban area wide-baseline images (e.g. unmanned aerial vehicle images and oblique images) with viewpoint change and occlusion.

Key words: urban area wide-baseline image, feature point matching, structure adaptive feature region, viewpoint variation, matching expansion

CLC Number:

P237

CHEN Min, ZHU Qing, HE Haiqing, YAN Shaohua, ZHAO Yitao. Structureadaptive feature point matching for urban area wide-baseline images with viewpoint variation[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(9): 1129-1140.

References

[1] 戴激光, 宋伟东, 贾永红, 等. 一种新的异源高分辨率光学卫星遥感影像自动匹配算法[J]. 测绘学报, 2013, 42(1):80-86. DAI Jiguang, SONG Weidong, JIA Yonghong, et al. A new automatically matching algorithm for multi-source high resolution optical satellite images[J]. Acta Geodaetica et Cartographica Sinica, 2013, 42(1):80-86.
[2] 叶沅鑫, 单杰, 彭剑威, 等. 利用局部自相似进行多光谱遥感图像自动配准[J]. 测绘学报, 2014, 43(3):268-275. DOI:10.13485/j.cnki.11-2089.2014.0039. YE Yuanxin, SHAN Jie, PENG Jianwei, et al. Automated multispectral remote sensing image registration using local self-similarity[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(3):268-275. DOI:10.13485/j.cnki.11-2089.2014.0039.
[3] 岳春宇, 江万寿. 几何约束和改进SIFT的SAR影像和光学影像自动配准方法[J]. 测绘学报, 2012, 41(4):570-576. YUE Chunyu, JIANG Wanshou. An automatic registration algorithm for SAR and optical images based on geometry constraint and improved SIFT[J]. Acta Geodaetica et Cartographica Sinica, 2012, 41(4):570-576.
[4] 陈敏, 朱庆, 朱军, 等. SAR影像与光学影像的高斯伽玛型边缘强度特征匹配法[J]. 测绘学报, 2016, 45(3):318-325. DOI:10.11947/j.AGCS.2016.20150084. CHEN Min, ZHU Qing, ZHU Jun, et al. Feature matching for SAR and optical images based on Gaussian-Gamma-shaped edge strength map[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(3):318-325. DOI:10.11947/j.AGCS.2016.20150084.
[5] 梁焕青, 谢意, 付四洲. 颜色不变量与AKAZE特征相结合的无人机影像匹配算法[J]. 测绘学报, 2017, 46(7):900-909. DOI:10.11947/j.AGCS.2017.20160436. LIANG Huanqing, XIE Yi, FU Sizhou. UAV image registration algorithm using color invariant and AKAZE feature[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(7):900-909. DOI:10.11947/j.AGCS.2017.20160436.
[6] ZITOVÁ B, FLUSSER J. Image registration methods:a survey[J]. Image and Vision Computing, 2003, 21(11):977-1000.
[7] XIONG Zhen, ZHANG Yun. A novel interest-point-matching algorithm for high-resolution satellite images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(12):4189-4200.
[8] GRUEN A. Development and status of image matching in photogrammetry[J]. The Photogrammetric Record, 2012, 27(137):36-57.
[9] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
[10] BAY H, ESS A, TUYTELAARS T, et al. Speeded-up robust features (SURF)[J]. Computer Vision and Image Understanding, 2008, 110(3):346-359.
[11] TOLA E, LEPETIT V, FUA P. Daisy:an efficient dense descriptor applied to wide-baseline stereo[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(5):815-830.
[12] MOREL J M, YU Guoshen. ASIFT:a new framework for fully affine invariant image comparison[J]. SIAM Journal on Imaging Sciences, 2009, 2(2):438-469.
[13] HU Han, ZHU Qing, DU Zhiqiang, et al. Reliable spatial relationship constrained feature point matching of oblique aerial images[J]. Photogrammetric Engineering & Remote Sensing, 2015, 81(1):49-58.
[14] 肖雄武, 郭丙轩, 李德仁, 等. 一种具有仿射不变性的倾斜影像快速匹配方法[J]. 测绘学报, 2015, 44(4):414-421. DOI:10.11947/j.AGCS.2015.20140048. XIAO Xiongwu, GUO Bingxuan, LI Deren, et al. A quick and affine invariance matching method for oblique images[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(4):414-421. DOI:10.11947/j.AGCS.2015.20140048.
[15] 肖雄武, 李德仁, 郭丙轩, 等. 一种具有视点不变性的倾斜影像快速匹配方法[J]. 武汉大学学报(信息科学版), 2016, 41(9):1151-1159. XIAO Xiongwu, LI Deren, GUO Bingxuan, et al. A robust and rapid viewpoint-invariant matching method for oblique images[J]. Geomatics and Information Science of Wuhan University, 2016, 41(9):1151-1159.
[16] 闫利, 费亮, 叶志云, 等. 大范围倾斜多视影像连接点自动提取的区域网平差法[J]. 测绘学报, 2016, 45(3):310-317. DOI:10.11947/j.AGCS.2016.20140673. YAN Li, FEI Liang, YE Zhiyun, et al. Automatic tie-points extraction for triangulation of large-scale oblique multi-view images[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(3):310-317. DOI:10.11947/j.AGCS.2016.20140673.
[17] 张力, 艾海滨, 许彪, 等. 基于多视影像匹配模型的倾斜航空影像自动连接点提取及区域网平差方法[J]. 测绘学报, 2017, 46(5):554-564. DOI:10.11947/j.AGCS.2017.20160571. ZHANG Li, AI Haibin, XU Biao, et al. Automatic tie-point extraction based on multiple-image matching and bundle adjustment of large block of oblique aerial images[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(5):554-564. DOI:10.11947/j.AGCS.2017.20160571.
[18] SUN Yanbiao, ZHAO Liang, HUANG Shoudong, et al. L2-SIFT:SIFT feature extraction and matching for large images in large-scale aerial photogrammetry[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014(91):1-16.
[19] JIANG San, JIANG Wanshou. On-board GNSS/IMU assisted feature extraction and matching for oblique UAV images[J]. Remote Sensing, 2017, 9(8):813.
[20] YU Yinan, HUANG Kaiqi, CHEN Wei, et al. A novel algorithm for view and illumination invariant image matching[J]. IEEE Transactions on Image Processing, 2012, 21(1):229-240.
[21] 仇春平, 于瑞鹏, 丁翠, 等. 面向倾斜立体影像的尺度不变特征匹配[J]. 遥感信息, 2016, 31(1):43-47. QIU Chunping, YU Ruipeng, DING Cui, et al. Oblique stereo image matching based on scale invariant feature[J]. Remote Sensing Information, 2016, 31(1):43-47.
[22] FISCHLER M A, BOLLES R C. Random sample consensus:a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6):381-395.
[23] LI Kai, YAO Jian. Line segment matching and reconstruction via exploiting coplanar cues[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017(125):33-49.
[24] CHEN Min, QIN Rongjun, HE Haiqing, et al. A local distinctive features matching method for remote sensing images with repetitive patterns[J]. Photogrammetric Engineering & Remote Sensing, 2018, 84(8):513-524.
[25] MIKOLAJCZYK K, SCHMID C. Scale & affine invariant interest point detectors[J]. International Journal of Computer Vision, 2004, 60(1):63-86.
[26] MATAS J, CHUM O, URBAN M, et al. Robust wide-baseline stereo from maximally stable extremal regions[J]. Image and Vision Computing, 2004, 22(10):761-767.
[27] HARRIS C, STEPHENS M. A combined corner and edge detector[C]//Proceedings of the 4th Alvey Vision Conference. Manchester, Britain:[s.n.], 1988:147-152.
[28] VON GIOI R G, JAKUBOWICZ J, MOREL J M, et al. LSD:a fast line segment detector with a false detection control[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(4):722-732.
[29] WU Changchang. SiftGPU:a GPU implementation of scale invariant feature transform[EB/OL]. (2011). http://ccwu.me/code.html.
[30] YU Guoshen, MOREL J M. ASIFT:an algorithm for fully affine invariant comparison[J]. Image Processing On Line, 2011(1):11-38.