Power tower detection in remote sensing imagery based on deformable network and transfer learning

doi:10.11947/j.AGCS.2020.20190356

Abstract

Abstract: Power towers are important parts of power infrastructure, and it is indispensable to detect them. In view of the low precision and poor result of detection algorithms for power towers in remote sensing imagery, this study improves Faster R-CNN based on deformable network and transfer learning. And then we propose a new detection framework for power tower in remote sensing imagery. The framework includes a feature extraction sub-network and an object detection sub-network. The feature extraction sub-network uses deformable network model, which reconstructs the convolutional layer, to improve the model's feature extraction ability of the power towers with geometric deformation. The model parameters obtained from the feature extraction sub-network training are transferred to object detection sub-network, which accurately obtains position of power towers through RPN network,deformable area pooling and nms algorithms. Finally, the object detection sub-network is finely tuned and achieve high-precision detection for power towers in remote sensing image. The results show that in the test datasets AP_0.5, AP_0.6 and ACC are 0.886 1, 0.839 6, 0.894 8 which are at least higher 0.2 than SSD YOLOv3, Faster R-CNN. It can be seen from the comparative experiment that this method for power towers detection has great application potential.

Key words: object detection, remote sensing imagery, deformable network, transfer learning, Faster R-CNN

CLC Number:

P227

ZHENG Xin, PAN Bin, ZHANG Jian. Power tower detection in remote sensing imagery based on deformable network and transfer learning[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(8): 1042-1050.

References

[1] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, CA:IEEE, 2005:886-893.
[2] 方路平, 何杭江, 周国民. 目标检测算法研究综述[J]. 计算机工程与应用, 2018, 54(13):11-18, 33. FANG Luping, HE Hangjiang, ZHOU Guomin. Research overview of object detection methods[J]. Computer Engineering and Applications, 2018, 54(13):11-18, 33.
[3] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
[4] BAY H, ESS A, TUYTELAARS T, et al. Speeded-up robust features (SURF)[J]. Computer Vision and Image Understanding, 2008, 110(3):346-359.
[5] 罗楠, 孙权森, 耿蕾蕾, 等. 一种扩展SURF描述符及其在遥感图像配准中的应用[J]. 测绘学报, 2013, 42(3):383-388. LUO Nan, SUN Quansen, GENG Leilei, et al. An extended SURF descriptor and its application in remote sensing images registration[J]. Acta Geodaetica et Cartographica Sinica, 2013, 42(3):383-388.
[6] 杨钊霞, 邹峥嵘, 陶超, 等. 空-谱信息与稀疏表示相结合的高光谱遥感影像分类[J]. 测绘学报, 2015, 44(7):775-781. DOI:10.11947/j.AGCS.2015.20140207. YANG Zhaoxia, ZOU Zhengrong, TAO Chao, et al. Hyperspectral image classification based on the combination of spatial-spectral feature and sparse representation[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(7):775-781. DOI:10.11947/j.AGCS.2015.20140207.
[7] 慎利, 唐宏, 王世东, 等. 结合空间像素模板和Adaboost算法的高分辨率遥感影像河流提取[J]. 测绘学报, 2013, 42(3):344-350. SHEN Li, TANG Hong, WANG Shidong, et al. River extraction from the high resolution remote sensing images based on spatially correlated pixels template and Adaboost algorithm[J]. Acta Geodaetica et Cartographica Sinica, 2013, 42(3):344-350.
[8] 叶利华, 王磊, 张文文, 等. 高分辨率光学遥感场景分类的深度度量学习方法[J]. 测绘学报, 2019, 48(6):698-707. DOI:10.11947/j.AGCS.2019.20180434. YE Lihua, WANG Lei, ZHANG Wenwen, et al. Deep metric learning method for high-resolution remote sensing image scene classification[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6):698-707. DOI:10.11947/j.AGCS.2019.20180434.
[9] GIRSHICK R, DONAHUE J, DARRELL T, et al. Region-based convolutional networks for accurate object detection and segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(1):142-158.
[10] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe:Curran Associates Inc., 2012:1097-1105.
[11] UIJLINGS J R R, VAN DE SANDE K E A, GEVERS T, et al. Selective search for object recognition[J]. International Journal of Computer Vision, 2013, 104(2):154-171.
[12] 殷文斌, 王成波, 袁翠, 等. 一种飞机目标的遥感识别方法[J]. 测绘通报, 2017(3):34-37. DOI:10.13474/j.cnki.11-2246.2017.0079. YIN Wenbin, WANG Chengbo, YUAN Cui, et al. A method of aircraft recognition in remote sensing images[J]. Bulletin of Surveying and Mapping, 2017(3):34-37. DOI:10.13474/j.cnki.11-2246.2017.0079.
[13] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9):1904-1916.
[14] 杨争艳. 基于非下采样轮廓波SPP Net的高分辨SAR图像变化检测[D]. 西安:西安电子科技大学, 2018. YANG Zhengyan. Change detection for high-resolution SAR images based on NSCT SPP net[D]. Xi'an:Xidian University, 2018.
[15] GIRSHICK R. Fast R-CNN[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile:IEEE, 2015:1440-1448.
[16] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149.
[17] 史晋涛, 李喆, 顾超越, 等. 基于样本扩充的Faster R-CNN电网异物监测技术[J]. 电网技术, 2020, 44(1):44-51. SHI Jintao, LI Zhe, GU Chaoyue, et al. Research on foreign matter monitoring of power grid with Faster R-CNN based on sample expansion[J]. Power System Technology, 2020, 44(1):44-51.
[18] PAN S J, YANG Qiang. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10):1345-1359.
[19] ROTHE R, GUILLAUMIN M, VAN GOOL L. Non-maximum suppression for object detection by passing messages between windows[C]//Proceedings of the 12th Asian Conference on Computer Vision. Singapore:Springer, 2015:290-306.
[20] DAI Jifeng, QI Haozhi, XIONG Yuwen. Deformable convolutional networks[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy:IEEE, 2017:764-773.
[21] HE Kaiming, ZHANG Xiangyu, REN Shaoping, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, Nevada:IEEE, 2016:770-778.
[22] 邓志鹏, 孙浩, 雷琳, 等. 基于多尺度形变特征卷积网络的高分辨率遥感影像目标检测[J]. 测绘学报, 2018, 47(9):1216-1227. DOI:10.11947/j.AGCS.2018.20170595. DENG Zhipeng, SUN Hao, LEI Lin, et al. Object detection in remote sensing imagery with multi-scale deformable convolutional networks[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(9):1216-1227. DOI:10.11947/j.AGCS.2018.20170595.
[23] 左宗成, 张文, 张东映. 融合可变形卷积与条件随机场的遥感影像语义分割方法[J]. 测绘学报, 2019, 48(6):718-726. DOI:10.11947/j.AGCS.2019.20170740. ZUO Zongcheng, ZHANG Wen, ZHANG Dongying. A remote sensing image semantic segmentation method by combining deformable convolution with conditional random fields[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6):718-726. DOI:10.11947/j.AGCS.2019.20170740.
[24] 李策, 张栋, 杜少毅, 等. 一种迁移学习和可变形卷积深度学习的蝴蝶检测算法[J]. 自动化学报, 2019, 45(9):1772-1782. LI Ce, ZHANG Dong, DU Shaoyi, et al. A butterfly detection algorithm based on transfer learning and deformable convolution deep learning[J]. Acta Automatica Sinica, 2019, 45(9):1772-1782.
[25] LIU Wei, ANGUELOV D, ERHAN D, et al. SSD:single shot multibox detector[C]//Proceedings of the 13th European Conference on Computer Vision. Amsterdam, the Netherlands:Springer, 2016:21-37.