航拍图像跨数据域特征迁移道路提取方法

doi:10.11947/j.AGCS.2020.20190274

测绘学报 ›› 2020, Vol. 49 ›› Issue (5): 611-621.doi: 10.11947/j.AGCS.2020.20190274

航拍图像跨数据域特征迁移道路提取方法

王舒洋¹, 慕晓冬¹, 贺浩², 杨东方², 马晨晖¹

1. 火箭军工程大学作战保障学院, 陕西西安 710025;
2. 火箭军工程大学导弹工程学院, 陕西西安 710025

收稿日期:2019-06-28 修回日期:2019-12-23 发布日期:2020-05-23
通讯作者: 贺浩 E-mail:hehao209@126.com
作者简介:王舒洋(1991-),女,博士生,研究方向为遥感图像处理与计算机视觉。E-mail:yelvlanshu@163.com
基金资助:
国家自然科学基金（61403398；61673017）；陕西省自然科学基金面上项目（2017JM6077）

Feature-representation-transfer based road extraction method for cross-domain aerial images

WANG Shuyang¹, MU Xiaodong¹, HE Hao², YANG Dongfang², MA Chenhui¹

1. The Rocket Force University of Engineering, College of Operational Support, Xi'an 710025, China;
2. The Rocket Force University of Engineering, College of Missile Engineering, Xi'an 710025, China

Received:2019-06-28 Revised:2019-12-23 Published:2020-05-23
Supported by:
The National Nature Science Foundation of China (Nos. 61403398;61673017);The General Project of Shaanxi Nature Science Foundation (No. 2017JM6077)

摘要/Abstract

摘要： 针对传统道路提取方法应用于新数据泛化能力不足的问题，研究了通过特征迁移和编解码网络实现跨数据域的道路提取方法。首先，构建了基于编解码网络的道路提取基本模型，用于实现单一数据来源的道路提取任务。然后，基于道路提取网络结构和循环一致性原则，提出了用于跨数据域图像特征迁移的循环生成对抗网络，使目标域图像映射入源域特征空间。使用预训练的道路提取模型处理特征迁移后的目标域图像，即可实现跨数据域道路提取任务。试验结果表明，本文所提方法能够拓展道路提取网络的泛化能力，准确有效地提取跨数据域图像中的道路目标。相较于未特征迁移的结果，本文所提方法大幅改善了道路提取指标，使得F1提升了50%以上。本文方法不需要目标域的标注信息，也不需要对道路提取网络进行微调训练，而只需训练由目标域向源域的特征迁移模型，所耗时间和人力成本较低，因而具有良好的应用价值。

关键词: 道路提取, 遥感, 迁移学习, 深度学习, 生成对抗网络, 编解码网络

Abstract: Aiming at the problem of the insufficient generalization ability of traditional road extraction methods when applying to a new dataset, this paper proposes a cross-domain road extraction method that realized by feature-representation-transfer and encoder-decoder network. Firstly, a basic road extraction model based on encoder-decoder network is designed to segment the road from a single data source. Then, based on the structure of road extraction network and the principle of cycle-consistent, a cycle generative adversarial network for feature transfer of cross-domain imagery is used, which maps the feature of target city images to the domain of source data. Finally, the pre-trained road extraction model is used to segment the target domain images after the feature transfer, so that the cross-domain road extraction can be realized. The experimental results show that the proposed method improves the generalization ability of the road extraction network and can extract the road target from cross-domain images accurately and effectively. Compared with the results without feature transfer, the proposed method greatly improves the road extraction metric, and increases the F1-score by more than 50%. The proposed method does not require any annotation of the target domain images, nor does it need to fine-tune the road extraction network, while it only need to train the feature transfer model from the target domain to the source domain. Therefore, it has good application value.

Key words: road extraction, remote sensing, transfer learning, deep learning, generative adversarial network, encoder-decoder network

中图分类号:

P237

王舒洋, 慕晓冬, 贺浩, 杨东方, 马晨晖. 航拍图像跨数据域特征迁移道路提取方法[J]. 测绘学报, 2020, 49(5): 611-621.

WANG Shuyang, MU Xiaodong, HE Hao, YANG Dongfang, MA Chenhui. Feature-representation-transfer based road extraction method for cross-domain aerial images[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(5): 611-621.

参考文献

[1] 吴亮, 胡云安. 遥感图像自动道路提取方法综述[J]. 自动化学报, 2010, 36(7): 912-922. WU Liang, HU Yun’an. A survey of automatic road extraction from remote sensing images[J]. Acta Automatica Sinica, 2010, 36(7): 912-922.
[2] MNIH V, HINTON G E. Learning to detect roads in high-resolution aerial images[C]//Proceedings of European Conference on Computer Vision: Part VI, September 5-11, 2010, Heraklion, Crete, Greece. Heraklion: Springer, 210-223.
[3] MNIH V. Machine learning for aerial image labeling[D]. Toronto: University of Toronto, 2013.
[4] WANG Jun, SONG Jingwei, CHEN Mingquan, et al. Road network extraction: a neural-dynamic framework based on deep learning and a finite state machine[J]. International Journal of Remote Sensing, 2015, 36(12): 3144-3169.
[5] ALSHEHHI R, MARPU P R, WOON W L, et al. Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 130: 139-149.
[6] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition, June 7-12, 2015, Boston, MA. Boston: IEEE, 2015: 3431-3440.
[7] ZHONG Zilong, LI J, CUI Weihong, et al. Fully convolutional networks for building and road extraction: preliminary results[C]//Proceedings of 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), July 10-15, 2016, Beijing, China. Beijing: IEEE, 2016: 1591-1594.
[8] WEI Yanan, WANG Zulin, XU Mai. Road structure refined CNN for road extraction in aerial image[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(5): 709-713.
[9] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer International Publishing, 2015: 234-241.
[10] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
[11] 贺浩, 王仕成, 杨东方, 等. 基于Encoder-Decoder网络的遥感影像道路提取方法[J]. 测绘学报, 2019, 48(3): 330-338. DOI: 10.11947/j.AGCS.2019.20180005. HE Hao, WANG Shicheng, YANG Dongfang, et al. A road extraction method for remote sensing image based on Encoder-Decoder network[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(3): 330-338. DOI: 10.11947/j.AGCS.2019.20180005.
[12] PANBOONYUEN T, JITKAJORNWANICH K, LAWAWIROJWONG S, et al. Road segmentation of remotely-sensed images using deep convolutional neural networks with landscape metrics and conditional random fields[J]. Remote Sensing, 2017, 9(7): 9070680.
[13] SUN Tao, CHEN Zehui, YANG Wenxiang, et al. Stacked U-nets with multi-output for road extraction[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), June 18-22, 2018, Salt Lake City, UT. Salt Lake City: IEEE Computer Society, 2018.
[14] HE H, WANG S C, YANG D F, et al. Light encoder-decoder network for road extraction of remote sensing images[J]. Journal of Applied Remote Sensing. 2019, 13(3): 034510.
[15] LI Yunsheng, YUAN Lu, VASCONCELOS N. Bidirectional learning for domain adaptation of semantic segmentation[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 1-8, 2019, Long Beach, USA. Long Beach: IEEE, 1-10.
[16] TSAI Y H, HUNG W C, SCHULTER S, et al. Learning to adapt structured output space for semantic segmentation[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 18-23, 2018, Salt Lake City, UT, USA. Salt Lake City: IEEE, 2018.
[17] HE Hao, YANG Dongfang, WANG Shicheng, et al. Road extraction by using atrous spatial pyramid pooling integrated encoder-decoder network and structural similarity loss[J]. Remote Sensing, 2019, 11(9): 1015.
[18] CLEVERT D A, UNTERTHINER T, HOCHREITER S. Fast and accurate deep network learning by exponential linear units (ELUs)[C]//Proceedings of the 4th International Conference on Learning Representations. San Juan, Puerto Rico: [s.n.], 2016.
[19] IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]// Proceedings of the 32nd International Conference on Machine Learning, July 6-11, 2015, Lille, France. Lille: JMLR, 2015: 448-456.
[20] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation. arXiv: 1706.05587, 2017.
[21] ZHU Junyan, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of IEEE International Conference on Computer Vision, October 22-29, 2017, Venice, Italy. Venice: IEEE, 2017: 2242-2251.
[22] ISOLA P, ZHU Junyan, ZHOU Tinghui, et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, July 21-26, 2017, Honolulu, USA. Honolulu: IEEE, 2017: 5967-5976.
[23] COSTEA D, LEORDEANU M. Aerial image geolocalization from recognition and matching of roads and intersections[C]//Proceedings of British Machine Vision Conference, September 19-22, 2016, York, UK. York: BMVA, 1-12.
[24] WIEDEMANN C, HEIPKE C, MAYER H, et al. Empirical evaluation of automatically extracted road axes[C]//Proceedings of the 9th Australasian Remote Sensing Photogrammetry Conference. Sydney: The University of New South Wales, 1998: 172-187.
[25] KINGMA D P, BA J. Adam: a method for stochastic optimization[C]//Proceedings of the 3rd International Conference on Learning Representations. CA, San Diego: [s.n.], 2015.
[26] GONG Jianya. Photogrammetry and deep learning[J]. Journal of Geodesy and Geoinformation Science, 2018, 1(1): 1-15 doi:10.11947/j.JGGS.2018.0101.

航拍图像跨数据域特征迁移道路提取方法

Feature-representation-transfer based road extraction method for cross-domain aerial images

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	张祖勋, 姜慧伟, 庞世燕, 胡翔云. 多时相遥感影像的变化检测研究现状与展望[J]. 测绘学报, 2022, 51(7): 1091-1107.
[2]	王桥. 地表异常遥感探测与即时诊断方法研究框架[J]. 测绘学报, 2022, 51(7): 1141-1152.
[3]	张克非, 李浩博, 王晓明, 朱丹彤, 何琦敏, 李龙江, 胡安东, 郑南山, 李怀展. 地基GNSS大气水汽探测遥感研究进展和展望[J]. 测绘学报, 2022, 51(7): 1172-1191.
[4]	张良培, 何江, 杨倩倩, 肖屹, 袁强强. 数据驱动的多源遥感信息融合研究进展[J]. 测绘学报, 2022, 51(7): 1317-1337.
[5]	吴立新, 齐源, 毛文飞, 刘善军, 丁逸凡, 荆凤, 申旭辉. 多波段多极化被动微波遥感地震应用研究进展与前沿方向探索[J]. 测绘学报, 2022, 51(7): 1356-1371.
[6]	张兵, 杨晓梅, 高连如, 孟瑜, 孙显, 肖晨超, 倪丽. 遥感大数据智能解译的地理学认知模型与方法[J]. 测绘学报, 2022, 51(7): 1398-1415.
[7]	许强, 朱星, 李为乐, 董秀军, 戴可人, 蒋亚楠, 陆会燕, 郭晨. “天-空-地”协同滑坡监测技术进展[J]. 测绘学报, 2022, 51(7): 1416-1436.
[8]	程涛, 张洋, James Haworth. 基于网络和图的时空智能——概念、方法和应用[J]. 测绘学报, 2022, 51(7): 1629-1639.
[9]	李德仁, 王密, 杨芳. 新一代智能测绘遥感科学试验卫星珞珈三号01星[J]. 测绘学报, 2022, 51(6): 789-796.
[10]	龚健雅, 宦麟茜, 郑先伟. 影像解译中的深度学习可解释性分析方法[J]. 测绘学报, 2022, 51(6): 873-884.
[11]	张勤, 赵超英, 陈雪蓉. 多源遥感地质灾害早期识别技术进展与发展趋势[J]. 测绘学报, 2022, 51(6): 885-896.
[12]	李荣兴, 李国君, 冯甜甜, 沈强, 乔刚, 叶真, 夏梦莲. 基于光学遥感卫星影像的南极冰流速产品和方法研究综述[J]. 测绘学报, 2022, 51(6): 953-963.
[13]	程结海, 黄中意, 王建如, 何湜. 高空间分辨率遥感影像最优分割结果自动确定方法[J]. 测绘学报, 2022, 51(5): 658-667.
[14]	白坤, 慕晓冬, 陈雪冰, 朱永清, 尤轩昂. 融合半监督学习的无监督遥感影像场景分类[J]. 测绘学报, 2022, 51(5): 691-702.
[15]	龚健雅, 张觅, 胡翔云, 张展, 李彦胜, 姜良存. 智能遥感深度学习框架与模型设计[J]. 测绘学报, 2022, 51(4): 475-487.