Aircraft detection in remote sensing images using cascade convolutional neural networks

doi:10.11947/j.AGCS.2019.20180471

Abstract

Abstract: Traditional aircraft detection algorithms which adopt handcraft features have poor performance in complex scene images and recognizing multi-scale objects. Methods using deep convolutional neural networks still face difficulty in dim small target search and recognition in large images with complex background. Aiming at these problems, a coarse-to-fine algorithm for aircraft detection in remote sensing images using cascade convolutional neural networks is proposed. To quickly and effectively acquire suspicious regions of interest (ROI), the whole image is searched by a small and shallow fully convolutional neural network which could deal with images of any size. Then deeper convolutional neural networks are used to refine the classification and location of the ROIs. A multilayer perceptron is introduced to the convolutional layer to improve identification capability of the convolutional neural networks and the strategies of multi-task learning and offline hard example mining are adopted in the process of training. At the detecting stage, the image pyramid is constructed and the redundant windows could be eliminated by the non-maximal suppression. Multiple datasets are tested and the results show that the proposed method has higher accuracy and stronger robustness and provides a fast and efficient solution for object detection in large remote sensing images.

Key words: aircraft detection, remote sensing image, cascade convolutional neural networks, hard example mining, deep learning

CLC Number:

P237

YU Donghang, GUO Haitao, ZHANG Baoming, ZHAO Chuan, LU Jun. Aircraft detection in remote sensing images using cascade convolutional neural networks[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(8): 1046-1058.

References

[1] 邓志鹏, 孙浩, 雷琳, 等. 基于多尺度形变特征卷积网络的高分辨率遥感影像目标检测[J]. 测绘学报, 2018, 47(9):1216-1227. DOI:10.11947/j.AGCS.2018.20170595. DENG Zhipeng, SUN Hao, LEI Lin, et al. Object detection in remote sensing imagery with multi-scale deformable convolutional networks[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(9):1216-1227. DOI:10.11947/j.AGCS.2018.20170595.
[2] 曾接贤, 付俊, 符祥. 特征点和不变矩结合的遥感图像飞机目标识别[J]. 中国图象图形学报, 2014, 19(4):592-602. ZENG Jiexian, FU Jun, FU Xiang. Aircraft target recognition in remote sensing images based on distribution of the feature points and invariant moments[J]. Journal of Image and Graphics, 2014, 19(4):592-602.
[3] HSIEH J W, CHEN J M, CHUANG C H, et al. Aircraft type recognition in satellite images[J]. IEE Proceedings-Vision, Image and Signal Processing, 2005, 152(3):307-315.
[4] ZHAO An, FU Kun, SUN Hao, et al. An effective method based on ACF for aircraft detection in remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(5):744-748.
[5] LEI Zhen, FANG Tao, HUO Hong, et al. Rotation-invariant object detection of remotely sensed images based on texton forest and Hough voting[J]. IEEE Transactions on Geoscience and Remote Sensing, 2012, 50(4):1206-1217. DOI:10.1109/TGRS.2011.2166966.
[6] 王树国, 黄勇杰, 张生. 可见光图像中飞机目标的特征选择及提取[J]. 哈尔滨工业大学学报, 2010, 42(7):1056-1059. WANG Shuguo, HUANG Yongjie, ZHANG Sheng. Selection and extraction of features of aircraft in optical image[J]. Journal of Harbin Institute of Technology, 2010, 42(7):1056-1059.
[7] 王文胜. 宽幅光学遥感图像舰船飞机目标检测识别技术研究[D]. 长春:中国科学院大学(中国科学院长春光学精密机械与物理研究所), 2018. WANG Wensheng. Research on technology of ship and aircraft targets recognition from large-field optical remote sensing image[D]. Changchun:Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, 2018.
[8] WANG Wensheng, NIE Ting, FU Tianjiao, et al. A novel method of aircraft detection based on high-resolution panchromatic optical remote sensing images[J]. Sensors, 2017, 17(5):1047. DOI:10.3390/s17051047.
[9] LI Yu, SUN Xian, WANG Hongqi, et al. Automatic target detection in high-resolution remote sensing images using a contour-based spatial model[J]. IEEE Geoscience and Remote Sensing Letters, 2012, 9(5):886-890. DOI:10.1109/LGRS.2012.2183337.
[10] SUN Hao, SUN Xian, WANG Hongqi, et al. Automatic target detection in high-resolution remote sensing images using spatial sparse coding bag-of-words model[J]. IEEE Geoscience and Remote Sensing Letters, 2012, 9(1):109-113.
[11] BAI Xiao, ZHANG Huigang, ZHOU Jun. VHR object detection based on structural feature extraction and query expansion[J]. IEEE Transactions on Geoscience and Remote Sensing, 2014, 52(10):6508-6520. DOI:10.1109/TGRS.2013.2296782.
[12] 岳伍军. 基于光学遥感图像的飞机目标检测算法研究[D]. 成都:西南交通大学, 2014. YUE Wujun. Research on airplane and target detection algorithms in optical remote sensing imagery[D]. Chengdu:Southwest Jiaotong University, 2014.
[13] 吴喆, 曾接贤, 高琪琪. 显著图和多特征结合的遥感图像飞机目标识别[J]. 中国图象图形学报, 2017, 22(4):532-541. WU Zhe, ZENG Jiexian, GAO Qiqi. Aircraft target recognition in remote sensing images based on saliency images and multi-feature combination[J]. Journal of Image and Graphics, 2017, 22(4):532-541.
[14] ZHANG Libao, ZHANG Yingying. Airport detection and aircraft recognition based on two-layer saliency model in high spatial resolution remote-sensing images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(4):1511-1524. DOI:10.1109/JSTARS.2016.2620900.
[15] LONG Yang, GONG Yiping, XIAO Zhifeng, et al. Accurate object localization in remote sensing images based on convolutional neural networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(5):2486-2498. DOI:10.1109/TGRS.2016.2645610.
[16] 姚相坤, 万里红, 霍宏, 等. 基于多结构卷积神经网络的高分遥感影像飞机目标检测[J]. 计算机工程, 2017, 43(1):259-267. YAO Xiangkun, WAN Lihong, HUO Hong, et al. Airplane object detection in high resolution remote sensing imagery based on multi-structure convolutional neural network[J]. Computer Engineering, 2017, 43(1):259-267.
[17] 刘欢. 基于高分辨率光学遥感影像的特定目标检测算法研究[D]. 哈尔滨:哈尔滨工业大学, 2017. LIU Huan. Research on detection algorithm for specific objects in high-resolution optical remote sensing images[D]. Harbin:Harbin Institute of Technology, 2017.
[18] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge, MA:MIT Press, 2015:91-99.
[19] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once:unified, real-time object detection[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016:779-788.
[20] 洪韬. 基于光学遥感影像的特定目标检测及识别[D]. 成都:电子科技大学, 2018. HONG Tao. Specific object detection and recognition in optical remote sensing image[D]. Chengdu:University of Electronic Science and Technology of China, 2018.
[21] 辛鹏, 许悦雷, 唐红, 等. 全卷积网络多层特征融合的飞机快速检测[J]. 光学学报, 2018, 38(3):0315003. XIN Peng, XU Yuelei, TANG Hong, et al. Fast airplane detection based on multi-layer feature fusion of fully convolutional networks[J]. Acta Optica Sinica, 2018, 38(3):0315003.
[22] HAN Xiaobing, ZHONG Yanfei, ZHANG Liangpei. An efficient and robust integrated geospatial object detection framework for high spatial resolution remote sensing imagery[J]. Remote Sensing, 2017, 9(7):666. DOI:10.3390/rs9070666.
[23] RADOVIC M, ADARKWA O, WANG Qiaosong. Object recognition in aerial images using convolutional neural networks[J]. Journal of Imaging, 2017, 3(2):21. DOI:10.3390/jimaging3020021.
[24] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv:1409.1556, 2014.
[25] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016:770-778.
[26] LIU Wei, ANGUELOV D, ERHAN D, et al. SSD:single shot multibox detector[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam:Springer International Publishing, 2016:21-37.
[27] VIOLA P, JONES M. Rapid object detection using a boosted cascade of simple features[C]//Proceedings of 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Kauai, HI:IEEE, 2001.
[28] LI Haoxiang, LIN Zhe, SHEN Xiaohui, et al. A convolutional neural network cascade for face detection[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA:IEEE, 2015:5325-5334.
[29] ZHANG Kaipeng, ZHANG Zhanpeng, LI Zhifeng, et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016, 23(10):1499-1503. DOI:10.1109/LSP.2016.2603342.
[30] LIN Ming, CHEN Qiang, YAN Shuicheng. Network in network[J]. arXiv:1312.4400, 2013.
[31] SHRIVASTAVA A, GUPTA A, GIRSHICK R. Training region-based object detectors with online hard example mining[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016:761-769.
[32] ZHU Haigang, CHEN Xiaogang, DAI Weiqun, et al. Orientation robust object detection in aerial images using deep convolutional neural network[C]//Proceedings of 2015 IEEE International Conference on Image Processing. Quebec City, QC:IEEE, 2015:3735-3739. DOI:10.1109/ICIP.2015.7351502.
[33] XIA Guisong, BAI Xiang, DING Jian, et al. DOTA:a large-scale dataset for object detection in aerial images[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT:IEEE, 2018:3974-3983.
[34] REDMON J, FARHADI A. YOLO9000:better, faster, stronger[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI:IEEE, 2017:6517-6525.
[35] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice:IEEE, 2017:2999-3007.
[36] HOWARD A G, ZHU Menglong, CHEN Bo, et al. MobileNets:efficient convolutional neural networks for mobile vision applications[J]. arXiv:1704.04861, 2017.
[37] ZHANG Xiangyu, ZHOU Xinyu, LIN Mengxiao, et al. Shufflenet:an extremely efficient convolutional neural network for mobile devices[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT:IEEE, 2018:6848-6856.