A deformable feature pyramid network for ship detection from remote sensing images

doi:10.11947/j.AGCS.2020.20190117

Abstract

Abstract: As a carrier of maritime transportation, the accurate detection of ships is of great significance and value in marine environmental protection, marine fishery production management, maritime traffic and emergency disposal, and national defense security applications. In recent years, the remote sensing ship detection method based on CNN (convolutional neural network) is facing big challenges to adapt to small-scale ships with random orientation and morphological characteristics due to insufficient resolution of the final layer features and convolution fixed geometry, thus reducing the accuracy of object detection. In order to tackle this problem, a remote sensing ship detection method based on deformable feature pyramid network with multi-scale feature fusion. First, the architecture of feature pyramid network is adopted to detect small-scale ship object by using a bottom-up refinement process and multi-scale feature fusion. Then, by introducing the deformable convolution and RoI (region of interest) pooling module to adapt to the ship object with random orientation and morphological characteristics, the ship detection accuracy is further improved. Experiments on 40 000 remote sensing images and over 67 280 ship objects demonstrate that the proposed method performs better than CNN. The rate of recall, accuracy, and F₁-Score are 85.8%, 97.9% and 91.5%, respectively.

Key words: ship detection, feature pyramid networks, deformable convolution module, deformable RoI pooling module

CLC Number:

P237

DENG Ruizhe, CHEN Qihao, CHEN Qi, LIU Xiuguo. A deformable feature pyramid network for ship detection from remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(6): 787-797.

References 29

[1]	陈韬亦, 陈金勇, 赵和鹏. 基于Ecogniton的光学遥感图像舰船目标检测[J]. 无线电工程, 2013, 43(11):11-13, 22. CHEN Taoyi, CHEN Jinyong, ZHAO Hepeng. Ecognition-based ship detection on optical remote sensing images[J]. Radio Engineering of China, 2013, 43(11):11-13, 22.
[2]	CRISP D J. A ship detection system for RADARSAT-2 dual-pol multi-look imagery implemented in the ADSS[C]//Proceedings of 2013 International Conference on Radar. Adelaide, SA, Australia:IEEE, 2016:318-323.
[3]	CRISP David James. The state-of-the-art in ship detection in Synthetic Aperture Radar imagery[R]. Australian Government, Department of Defence,[s.n.], 2004.
[4]	FINGAS M F, BROWN C E. Review of ship detection from airborne platforms[J]. Canadian Journal of Remote Sensing, 2001, 27(4):379-385.
[5]	LENG Xiangguang, JI Kefeng, ZHOU Shilin, et al. An adaptive ship detection scheme for spaceborne SAR imagery[J]. Sensors,2016, 16(9):1345.
[6]	WANG Chonglei, BI Fukun, ZHANG Weiping, et al. An intensity-space domain CFAR method for ship detection in HR SAR images[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(4):529-533.
[7]	伍广明, 陈奇, SHIBASAKI R, 等. 基于U型卷积神经网络的航空影像建筑物检测[J]. 测绘学报, 2018, 47(6):864-872. DOI:10.11947/j.AGCS.2018.20170651. WU Guangming, CHEN Qi, SHIBASAKI R, et al. High precision building detection from aerial imagery using a U-net like convolutional architecture[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6):864-872. DOI:10.11947/j.AGCS.2018.20170651.
[8]	REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal:MIT Press, 2015:91-99.
[9]	LIU Wei, ANGUELOY D, ERHAN D, et al. SSD:single shot MultiBox detector[C]//Proceedings of the European Conference on Computer Vision. Amsterdam:Springer, 2016:21-37.
[10]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once:unified, real-time object detection[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA:IEEE, 2016:779-788.
[11]	DAI Jifeng, LI Yi, HE Kaiming, et al. R-FCN:Object detection via region-based fully convolutional networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. New York,United States:Curran Associates Inc., 2016:379-387.
[12]	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA:IEEE, 2017:936-944.
[13]	KAREN Simonyan, ANDREW Zisserman. Very deep convolutional networks for large-scale image recognition[C]//Proceedings of the International Conference on Learning Representation. San-diego, California,USA:[s.n.],2014.
[14]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, United States:IEEE, 2016:770-778.
[15]	ZHANG Ruiqing, YAO Jian, ZHANG Kao, et al. S-CNN ship detection from high-resolution remote sensing images[J]. ISPRS-International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016, XLI-B7:423-430.
[16]	LIU Yang, ZHANG Miaohui, XU Peng, et al. SAR ship detection using sea-land segmentation-based convolutional neural network[C]//Proceedings of International Workshop on Remote Sensing with Intelligent Processing. Shanghai, China:IEEE, 2017:1-4.
[17]	KANG Miao, LENG Xiangguang, LIN Zhao, et al. A modified faster R-CNN based on CFAR algorithm for SAR ship detection[C]//Proceedings of International Workshop on Remote Sensing with Intelligent Processing. Shanghai, China:IEEE, 2017:1-4.
[18]	KANG Miao, JI Kefeng, LENG Xiangguang, et al. Contextual region-based convolutional neural network with multilayer fusion for SAR ship detection[J]. Remote Sensing, 2017, 9(8):860.
[19]	TANG Jiexiong, DENG Chenwei, HUANG Guangbin, et al. Compressed-domain ship detection on spaceborne optical image using deep neural network and extreme learning machine[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(3):1174-1185.
[20]	高鑫, 李慧, 张义, 等. 基于可变形卷积神经网络的遥感影像密集区域车辆检测方法[J]. 电子与信息学报, 2018, 40(12):2812-2819. GAO Xin, LI Hui, ZHANG Yi, et al. Vehicle detection in remote sensing images of dense areas based on deformable convolution neural network[J]. Journal of Electronics and Information Technology, 2018, 40(12):2812-2819.
[21]	邓志鹏, 孙浩, 雷琳, 等. 基于多尺度形变特征卷积网络的高分辨率遥感影像目标检测[J]. 测绘学报, 2018, 47(9):1216-1227. DOI:10.11947/j.AGCS.2018.20170595. DENG Zhipeng, SUN Hao, LEI Lin, et al. Object detection in remote sensing imagery with multi-scale deformable convolutional networks[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(9):1216-1227. DOI:10.11947/j.AGCS.2018.20170595.
[22]	DAI Jifeng, QI Haozhi, XIONG Yuwen, et al. Deformable convolutional networks[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy:IEEE, 2017:764-773.
[23]	YANG Xue, SUN Hao, FU Kun, et al. Automatic ship detection in remote sensing images from Google earth of complex scenes based on multiscale rotation dense feature pyramid networks[J]. Remote Sensing, 2018, 10(1):132.
[24]	YANG Xue, SUN Hao, SUN Xian, et al. Position detection and direction prediction for arbitrary-oriented ships via multitask rotation region convolutional neural network[J]. IEEE Access, 2018, 6:50839-50849.
[25]	CHEN Chaoyue, GONG Weiguo, CHEN Yongliang, et al. Object detection in remote sensing images based on a scene-contextual feature pyramid network[J]. Remote Sensing, 2019, 11(3):339.
[26]	ABADI Martin, BARHAM Paul, CHEN Jianmin, et al. Tensorflow:large-scale machine learning on heterogeneous distributed systems[C]//Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation (OSDI'16), USENIX Association, USA:[s.n.], 265-283.
[27]	XIA Guisong, BAI Xiang, DING Jian, et al. DOTA:a large-scale dataset for object detection in aerial images[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA:IEEE, 2018:3974-3983.
[28]	SHRIVASTAVA A, GUPTA A, GIRSHICK R. Training region-based object detectors with online hard example mining[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA:IEEE, 2016:761-769.
[29]	POWERS D M W. Evaluation:From precision, recall and f-measure to ROC, informedness, markedness & correlation[J]. Journal of Machine Learning Technologies, 2011, 2(1):37-63.