The transfer learning with convolutional neural network method of side-scan sonar to identify wreck images

doi:10.11947/j.AGCS.2021.20200187

Abstract

Abstract: The Side-scan sonar image automatic recognition is an important part of verification for underwater obstacle and wreck search and rescue, in view of the traditional artificial interpretation of side-scan sonar image is inefficient, time consuming and resource consumption and strong subjective uncertainty and excessive reliance on experience. This paper attempts to introduce the method of convolutional neural network, considering that the side-scan sonar shipwreck image belongs to a small sample data set, and an automatic recognition method of side-scan sonar shipwreck image based on transfer learning is proposed.The sample data were expanded by means of normalization and image enhancement, the training set and testing set were divided into 4∶1 proportions, and an improved model was designed according to the characteristics of the side-scan sonar wreck data set by referring to the classical VGG-16 model, then, the improved model trained on the ImageNet image data set is used to learn and experiment on the small sample side-scan sonar shipwreck data set using two transfer learning methods: freeze and train and fine-tuning, and compared with new learning. The results show that the accuracy of the three methods for the recognition of side-scan sonar shipwreck images is 93.71%, 84.49% and 90.58%, respectively. The first transfer learning method has the highest accuracy rate, the fastest model convergence speed, and the highest AP value 92.45%, which is 8.06% and 3.06% higher than the second transfer learning and the new learning method, respectively,and has a better effect in improving the model’s recognition ability and training efficiency. which verifies the effectiveness and feasibility of this method and has certain practical guiding significance.

Key words: side-scan sonar wreck image, image recognition, transfer learning, convolutional neural network, VGG-16

CLC Number:

P227

TANG Yulin, JIN Shaohua, BIAN Gang, ZHANG Yonghou, LI Fan. The transfer learning with convolutional neural network method of side-scan sonar to identify wreck images[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(2): 260-269.

References

[1] LEHARDY P K, MOORE C. Deep ocean search for Malaysia airlines flight 370[C]//Proceedings of 2014 Ocean-St. John's. St. John's, NL, Canada:IEEE, 2014:1-4.
[2] 肖付民, 刘雁春, 暴景阳, 等. 海道测量学概论[M]. 2版. 北京:测绘出版社, 2016:1-3. XIAO Fumin, LIU Yanchun, BAO Jingyang, et al. Introduction for hydrography[M]. 2nd ed. Beijing:Surveying and Mapping Publishing House, 2016:1-3.
[3] 赵建虎, 李娟娟, 李萌. 海洋测量的进展及发展趋势[J]. 测绘信息与工程, 2009, 34(4):25-27. ZHAO Jianhu, LI Juanjuan, LI Meng. Progress and future trend of hydrographic surveying and charting[J]. Journal of Geomatics, 2009, 34(4):25-27.
[4] 曹惠芬. 我国深海探测技术装备发展现状[J]. 船舶物资与市场, 2005(2):19-22. CAO Huifen. Development status of deep-sea exploration technology and equipment in China[J]. Ship Supplies and Marketing, 2005(2):19-22.
[5] 王童豪, 彭星光, 潘光, 等. 无人水下航行器的发展现状与关键技术[J]. 宇航总体技术, 2017, 1(4):52-64. WANG Tonghao, PENG Xingguang, PAN Guang, et al. Development and key technologies of unmanned underwater vehicles[J]. Aerospace Systems Engineering Technology, 2017, 1(4):52-64.
[6] ZHANG Lihua, LIU Xianpeng, JIA Shuaidong, et al. A line-surface integrated algorithm for underwater terrain matching[J]. Journal of Geodesy and Geoinformation Science, 2009, 2(4):10-20.
[7] 刘晨晨, 李晨, 张之猛. 基于PCNN的声纳图像纹理特征提取[J]. 自动化仪表, 2009, 30(2):18-20. LIU Chenchen, LI Chen, ZHANG Zhimeng. PCNN-based texture feature extraction of sonar images[J]. Process Automation Instrumentation, 2009, 30(2):18-20.
[8] 刘卓夫, 桑恩方. 基于纹理的声纳图像识别[J]. 计算机工程, 2004, 30(14):113-115. LIU Zhuofu, SANG Enfang. Sonar image recognition based on texture[J]. Computer Engineering, 2004, 30(14):113-115.
[9] 李海滨, 滕惠忠, 宋海英, 等. 基于侧扫声纳图像海底目标物提取方法[J]. 海洋测绘, 2010, 30(6):71-73. LI Haibin, TENG Huizhong, SONG Haiying, et al. Technology on the extraction of seabed target based on high resolution side-scan sonar[J]. Hydrographic Surveying and Charting, 2010, 30(6):71-73.
[10] HE Hao, WANG Shuyang, WANG Shicheng, et al. A road extraction method for remote sensing image based on encoder-decoder network[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(2):16-25.
[11] 吕良, 周超烨, 陈春, 等. 基于虚警函数的侧扫声纳水下目标实时检测方法[J]. 海洋测绘, 2013, 33(4):35-38. LÜ Liang, ZHOU Chaoye, CHEN Chun, et al. Real-time detection of underwater target using Side-scan sonar based on false alarm function[J]. Hydrographic Surveying and Charting, 2013, 33(4):35-38.
[12] 朱殿尧, 卞红雨. 侧扫声纳目标自动探测研究[J]. 吉林大学学报(信息科学版), 2008, 26(6):627-631. ZHU Dianyao, BIAN Hongyu. Research of side-scan sonar target auto detection[J]. Journal of Jilin University (Information Science Edition), 2008, 26(6):627-631.
[13] DAI Yuchao, ZHANG Jing, HE Mingyi, et al. Salient object detection from multi-spectral remote sensing images with deep residual network[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(2):101-110.
[14] 田杰, 张春华. 基于分形的水声图像目标探测[J]. 中国图象图形学报, 2005, 10(4):479-483. TIAN Jie, ZHANG Chunhua. Fractal-based detection of objects in underwater images[J]. Journal of Image and Graphics, 2005, 10(4):479-483.
[15] 朱殿尧. 水声图像处理与模式识别[D]. 哈尔滨:哈尔滨工程大学, 2008. ZHU Dianyao. Underwater acoustic image processing and pattern recognition[D]. Harbin:Harbin Engineering University, 2008.
[16] KÜÇÜKBAYRAK M, GÜNEŞ Ö, ARICA N. Underwater acoustic signal recognition methods[J]. Journal of Naval Science and Engineering, 2009, 5(3):64-78.
[17] KAMAL S, MOHAMMED S K, SASEENDRAN PILLAI P R, et al. Deep learning architectures for underwater target recognition[C]//Proceeding of 2013 Ocean Electronics (SYMPOL). Kochi, India:IEEE, 2013:48-54.
[18] RHINELANDER J. Feature extraction and target classification of side-scan sonar images[C]//Proceedings of 2016 IEEE Symposium Series on Computational Intelligence (SSCI). Athens, Greece:IEEE, 2016:1-6.
[19] 郭军, 马金凤, 王爱学. 基于SVM算法和GLCM的侧扫声纳影像分类研究[J]. 测绘与空间地理信息, 2015, 38(3):60-63. GUO Jun, MA Jinfeng, WANG Aixue. Study of side scan sonar image classification based on SVM and gray level Co-Cccurrence Matrix[J]. Geomatics & Spatial Information Technology, 2015, 38(3):60-63.
[20] 陈强. 基于水声图像水下目标识别的技术研究[D]. 哈尔滨:哈尔滨工程大学, 2012. CHEN Qiang. Research-based underwater acoustic images underwater target recognition[D]. Harbin:Harbin Engineering University, 2012.
[21] 郭海涛. 高分辨率成像声呐后置图像处理[D]. 哈尔滨:哈尔滨工程大学, 2002(13):74-82. GUO Haitao. Study on image recognition technique of high resolution imaging sonar[D]. Harbin:Harbin Engineering University, 2002(13):74-82.
[22]LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of IEEE, 1998, 86(11):2278-2324.
[23]FUKUSHIMA K. Neocognitron:a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position[J]. Biological Cybernetics, 1980, 36(4):193-202.
[24] SUN Long, WU Tao, SUN Guangcai,et al. Object detection research of SAR image using improved faster region based convolutional neural network[J].Journal of Geodesy and Geoinformation Science,2020,3(03):18-28.
[25] 冯国徽. 基于卷积神经网络VGG模型的小规模图像分类[D].兰州:兰州大学,2018. FENG Guohui.Small-scale picture classification based on convolutional neural network VGG[D].Lanzhou:Lanzhou University,2018.
[26] REZA Fuad Rachmadi,I Ketut Eddy Purnama. Paralel spatial pyramid convolutional neural network untuk verifikasi kekerabatan berbasis citra wajah[J]. Jurnal Teknologi dan Sistem Komputer,2018,6(4):152-157.
[27] TIAN Lei, FAN Chunxiao, YUE Ming, et al. Stacked PCA Network (SPCANet):An effective deep learning for face recognition[C]//Proceedings of 2015 IEEE International Conference on Digital Signal Processing. Singapore:IEEE, 2015:1039-1043.
[28] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Image Net classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90.
[29] BENGIO Y, BASTIEN F, BERGERON A, et al. Deep learners benefit more from out-of-distribution examples[C]//Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. Ft. Lauderdale, USA:MIT Press, 2012:164-172.
[30] PAN S J, YANG Qiang. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10):1345-1359.
[31] BIAN Yingxian, GUO Genchang. A transfer learning and performance comparison of deep learning models for pedestrian classification under automobile driving environment[J]. The Journal of Korean Institute of Information Technology, 2018, 16(10):83-92.
[32] DO C B, NG A Y. Transfer learning for text classification[C]//Advances in Neural Information Processing Systems 18. Vancouver, B.C., Canada:Curran Associates, Inc., 2005:1391-1398.
[33] MIHALKOVA L, HUYNH T, MOONEY R J. Mapping and revising Markov logic networks for transfer[C]//Proceedings of the 22nd AAAI Conference on Artificial Intelligence (AAAI-2007). Vancouver, British Columbia, Canada:AAAI, 2007:608-614.
[34] NICULESCU-MIZIL A, CARUANA R. Inductive Transfer for Bayesian network structure learning[C]//Proceedings of 2012 ICML Workshop on Unsupervised and Transfer Learning. 2012, 27:167-181.
[35] YOSINSKI J, CLUNE J, BENGIO Y, et al. How transferable are features in deep neural networks?[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge, MA:MIT Press, 2014:3320-3328.
[36] 常东良. 基于深度学习的小样本图像分类方法研究[D]. 兰州:兰州理工大学, 2019. CHANG Dongliang. The research on small sample image classification method based on deep learning[D]. Lanzhou:Lanzhou University of Technology, 2019.
[37] 郑欣悦. 基于深度学习的少样本图像分类方法[D]. 北京:中国科学院大学(中国科学院国家空间科学中心), 2019. ZHENG Xinyue. Few-shot image classification method based on deep learning[D]. Beijing:University of Chinese Academy of Sciences (National Space Science Center, Chinese Academy of Sciences), 2019.
[38] 耿艳磊, 陶超, 沈靖, 等. 高分辨率遥感影像语义分割的半监督全卷积网络法[J]. 测绘学报, 2020, 49(4):499-508. DOI:10.11947/j.AGCS.2020.20190044. GENG Yanlei, TAO Chao, SHEN Jing, et al. High-resolution remote sensing image semantic segmentation based on semi-supervised full convolution network method[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(4):499-508. DOI:10.11947/j.AGCS.2020.20190044.
[39] FAN Dazhao, DONG Yang,ZHANG Yongsheng. Satellite image matching method based on deep convolutional neural network[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(2):90-100.
[40] LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. NewYork:IEEE Press, 2017:936-944.
[41] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems. Lake Tahoe, Nevada, USA:Morgan Kaufmann Publishers, Inc., 2012:1097-1105.