测绘学报 ›› 2024, Vol. 53 ›› Issue (2): 321-331.doi: 10.11947/j.AGCS.2024.20220636

• 摄影测量学与遥感 • 上一篇    下一篇

融合深度特征的无人机影像SfM重建

姜三1,2, 刘凯1, 李清泉2, 江万寿3   

  1. 1. 中国地质大学(武汉)计算机学院, 湖北 武汉 430074;
    2. 人工智能与数字经济广东省实验室(深圳), 广东 深圳 518060;
    3. 武汉大学测绘遥感信息工程国家重点实验室, 湖北 武汉 430079
  • 收稿日期:2022-11-08 修回日期:2024-01-10 发布日期:2024-03-08
  • 通讯作者: 李清泉 E-mail:liqq@szu.edu.cn
  • 作者简介:姜三(1987-),男,博士,副研究员,研究方向为多源影像匹配和三维重建的理论和方法。E-mail:jiangsan@cug.edu.cn
  • 基金资助:
    国家自然科学基金(42371442);湖北省自然科学基金(2023AFB568);人工智能与数字经济广东省实验室(深圳)开放课题资助(GML-KF-22-08)

Learned local features for SfM reconstruction of UAV images

JIANG San1,2, LIU Kai1, LI Qingquan2, JIANG Wanshou3   

  1. 1. School of Computer Science, China University of Geosciences, Wuhan 430074, China;
    2. Guangdong Laboratory of Artificial Intelligence and Digital Economy (Shenzhen), Shenzhen 518060, China;
    3. State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China
  • Received:2022-11-08 Revised:2024-01-10 Published:2024-03-08
  • Supported by:
    The National Natural Science Foundation of China (No.42371442); The Hubei Provincial Natural Science Foundation of China (No.2023AFB568); The Open Research Fund from the Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) (No.GML-KF-22-08)

摘要: 可靠特征匹配是无人机影像运动恢复结构(SfM)的重要环节。近年来,深度学习被用于特征提取和匹配,在基准数据集表现优于SIFT等手工特征。但是,公开模型往往采用互联网照片进行训练和测试,鲜有用于无人机影像SfM三维重建的性能评价。利用多组不同特点的无人机数据集,本文对比分析手工特征和深度学习特征在无人机影像特征匹配和SfM三维重建的综合性能。试验结果表明,利用公开的预训练模型,结合手工特征的高精度定位和深度学习的特征描述能力,可实现更准确和完整的特征匹配,并在SfM三维重建中取得与SIFT等手工特征相当,甚至更优的性能。

关键词: 摄影测量, 三维重建, 运动恢复结构, 深度特征, 卷积神经网络

Abstract: Reliable feature matching plays an essential role in SfM (structure from motion) for UAV (unmanned aerial vehicle) images. Recently, deep learning-based methods have been used for feature detection and matching, which outperforms traditional handcrafted methods, e.g., SIFT, on benchmark datasets. However, few studies have reported their performance on UAV images as these models are trained and tested using internet photos. By using UAV datasets with varying features, this study evaluated both handcrafted and learned methods in terms of feature matching and SfM-based image orientation. The experimental results show that even with the pretrained public-available models, more accurate and complete feature matching can be obtained through the combination of high-precision localization of handcrafted detectors and the high representation ability of learned descriptors, which has competitive or better performance in SfM-based image orientation when compared with SIFT-like handcrafted methods.

Key words: photogrammetry, 3D reconstruction, structure from motion, learned feature, convolutional neural network

中图分类号: