Salient Object Detection from Multi-spectral Remote Sensing Images with Deep Residual Network

doi:10.11947/j.AGCS.2018.20170633

Abstract

Abstract: This paper focuses on intelligent photogrammetry deep learning:deep residual method.Salient object detection aims at identifying the visually interesting object regions that are consistent with human perception.Multispectral remote sensing images provide rich radiometric information in revealing the physical properties of the observed objects,therefore promise a great potential in salient object detection tasks.Conventional salient object detection methods often employ handcrafted features to predict saliency by evaluating the pixel-wise or superpixel-wise similarity.With the recent emergence of deep learning based approaches,in particular,fully convolutional neural networks,there has been profound progress in visual saliency detection.However,this success has not been extended to multispectral remote sensing images,and existing multispectral salient object detection methods are still mainly based on handcrafted features,essentially due to the difficulties in image acquisition and labeling.In this paper,we propose a novel deep residual network based on a top-down model,which is trained in an end-to-end manner to tackle the above issues in multispectral salient object detection.Our model effectively exploits the saliency cues at different levels of the deep residual network.To overcome the limited availability of remote sensing images in training of our deep residual network,we also introduce a new spectral image reconstruction model that can generate multispectral images from RGB images.Our extensive experimental evaluations using both multispectral and RGB salient object detection datasets demonstrate a significant performance improvement of more than 10% compared with the state-of-the-art methods.

Key words: deep residual network, salient object detection, spectral super-resolution, top-down model, remote sensing image processing

CLC Number:

TP751

DAI Yuchao, ZHANG Jing, Fatih PORIKLI, HE Mingyi. Salient Object Detection from Multi-spectral Remote Sensing Images with Deep Residual Network[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6): 873-881.

References

[1] LI Junxia, MA Runing, DING Jundi. Saliency-seeded Region Merging:Automatic Object Segmentation[C]//First Asian Conference on Pattern Recognition. Beijing, China:IEEE, 2011:691-695.
[2] MARCHESOTTI L, CIFARELLI C, CSURKA G. A Framework for Visual Saliency Detection with Applications to Image Thumbnailing[C]//IEEE 12th International Conference on Computer Vision. Kyoto, Japan:IEEE, 2009:2232-2239.
[3] ZHANG Guoxin, CHENG Mingming, HU Shimin, et al. A Shape Preserving Approach to Image Resizing[J]. Computer Graphics Forum, 2009, 28(7):1897-1906.
[4] NAVALPAKKAM V, ITTI L. An Integrated Model of Top-down and Bottom-up Attention for Optimizing Detection Speed[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York, NY:IEEE, 2006:2049-2056.
[5] SHARMA S, KIROS R, SALAKHUTDINOV R. Action Recognition Using Visual Attention[C]//NIPS Time Series Workshop, 2015.
[6] SHELHAMER E, LONG J, DARRELL T. Fully Convolutional Networks for Semantic Segmentation[C]//IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA:IEEE, 2015:3431-3440.
[7] ZHU Wangjiang, LIANG Shuang, WEI Yichen, et al. Saliency Optimization from Robust Background Detection[C]//IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH:IEEE, 2014:2814-2821.
[8] JIANG Peng, LING Haibin, YU Jingyi, et al. Salient Region Detection by UFO:Uniqueness, Focusness and Objectness[C]//IEEE International Conference on Computer Vision. Sydney, NSW, Australia:IEEE, 2013:1976-1983.
[9] SHEN Xiaohui, WU Ying. A Unified Approach to Salient Object Detection via Low Rank Matrix Recovery[C]//IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI:IEEE, 2012:853-860.
[10] BORJI A, CHENG Mingming, JIANG Huaizu, et al. Salient Object Detection:A Benchmark[J]. IEEE Transactions on Image Processing, 2015, 24(12):5706-5722.
[11] LI Guanbin, YU Yizhou. Visual Saliency Based on Multiscale Deep Features[C]//IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA:IEEE, 2015:5455-5463.
[12] LI Guanbin, YU Yizhou. Visual Saliency Detection Based on Multiscale Deep CNN Features[J]. IEEE Transactions on Image Processing, 2016, 25(11):5012-5024.
[13] ZHANG Jing, LI Bo, DAI Yuchao, et al. Integrated Deep and Shallow Networks for Salient Object Detection[C]//IEEE International Conference on Image Processing. Beijing, China:IEEE, 2017:1537-1541.
[14] LI Xi, ZHAO Liming, WEI Lina, et al. Deepsaliency:Multi-task Deep Neural Network Model for Salient Object Detection[J]. IEEE Transactions on Image Processing, 2016, 25(8):3919-3930.
[15] LI Guanbin, YU Yizhou. Deep Contrast Learning for Salient Object Detection[C]//IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016:478-487.
[16] ZHANG Jing, DAI Yuchao, PORIKLI F. Deep Salient Object Detection by Integrating Multi-level Cues[C]//IEEE Winter Conference on Application of Computer Vision. Santa Rosa, CA:IEEE, 2017:1-10.
[17] HOU Qibin, CHENG Mingming, HU Xiaowei, et al. Deeply Supervised Salient Object Detection with Short Connections[C]//IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, Hawaii:IEEE, 2017:5300-5309.
[18] ZHANG Jing, DAI Yuchao, PORIKLI F, et al. Multi-Scale Salient Object Detection with Pyramid Spatial Pooling[C]//Asia-Pacific Signal and Information Processing Association Annual Summit and Conference. Kuala Lumpur, Malaysia:IEEE, 2017:1286-1291.
[19] ZHANG Ping, WANG Dong, LU Huchuan, et al. Amulet:Aggregating Multi-level Convolutional Features for Salient Object Detection[C]//IEEE International Conference on Computer Vision. Venice, Italy:IEEE, 2017:202-211.
[20] ZHANG Jing, DAI Yuchao, LI Bo, et al. Attention to the Scale:Deep Multi-scale Salient Object Detection[C]//International Conference on Digital Image Computing:Techniques and Application. Sydney, NSW, Australia:IEEE, 2017:1-7.
[21] ZHANG Dingwen, HAN Junwei, ZHANG Yu. Supervision by Fusion:Towards Unsupervised Learning of Deep Salient Object Detector[C]//IEEE International Conference on Computer Vision. Venice, Italy:IEEE, 2017:4068-4076.
[22] ZHANG Jing, ZHANG Tong, DAI Yuchao, et al. Deep Unsupervised Saliency Detection:A Multiple Noisy Labeling Perspective[C]//IEEE International Conference on Computer Vision.[s.l.]:IEEE, 2018.
[23] WANG Qi, YAN Pingkun, YUAN Yuan, et al. Multi-Spectral Saliency Detection[J]. Pattern Recognition Letters, 2013, 34(1):34-41.
[24] WANG Qi, ZHU Guokang, YUAN Yuan. Multi-Spectral Dataset and Its Application in Saliency Detection[J]. Computer Vision and Image Understanding, 2013, 117(12):1748-1754.
[25] 潘朝. 多尺度显著性引导的高分辨率遥感影像建筑物提取[J]. 科技创新与生产力, 2017(5):106-109. PAN Zhao. Building Extraction from High Resolution Remote Sensing Image Based on Visual Saliency Detection[J]. Technology Innovation and Productivity, 2017(5):106-109.
[26] 崔晓光. 融合目标信息的遥感图像显著性检测方法研究[D]. 北京:中国科学院大学, 2013. CUI Xiaoguang. Saliency Detection Methods via Fusing Target Information[D]. Beijing:University of Chinese Academy of Sciences, 2013.
[27] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep Residual Learning for Image Recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016:770-778.
[28] BROWN M, SUSSTRUNK S. Multi-spectral SIFT for Scene Category Recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI:IEEE, 2011:177-184.
[29] YU F, KOLTUN V. Multi-Scale Context Aggregation by Dilated Convolutions[C]//International Conference on Learning Representations.[s.l.]:IEEE, 2016.
[30] LIU Tie, SUN Jian, ZHENG Nanning, et al. Learning to Detect a Salient Object[C]//IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, MN:IEEE, 2007:1-8.
[31] JIANG Huaizu, WANG Jingdong, YUAN Zejiang, et al. Salient Object Detection:A Discriminative Regional Feature Integration Approach[C]//IEEE Conference on Computer Vision and Pattern Recognition. Portland, OR:IEEE, 2013:2083-2090.
[32] LI Xiaohui, LU Huchuan, ZHANG Lihe, et al. Saliency Detection via Dense and Sparse Reconstruction[C]//IEEE International Conference on Computer Vision. Sydney, NSW, Australia:IEEE, 2013:2976-2983.
[33] JIANG Bowen, ZHANG Lihe, LU Huchuan, et al. Saliency Detection via Absorbing Markov Chain[C]//IEEE International Conference on Computer Vision. Sydney, NSW, Australia:IEEE, 2013:1665-1672.
[34] WANG Lijun, LU Huchuan, RUAN Xiang, et al. Deep Networks for Saliency Detection via Local Estimation and Global Search[C]//IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA:IEEE, 2015:3183-3192.
[35] WANG Linzhao, WANG Lijun, LU Huchuan, et al. Saliency Detection with Recurrent Fully Convolutional Networks[M]//LEIBE B, MATAS J, SEBE N, et al. Computer Vision-ECCV 2016. Cham:Springer, 2016:825-841.