Unsupervised Remote Sensing Domain Adaptation Method with Adversarial Network and Auxiliary Task

doi:10.11947/j.AGCS.2017.20170291

Abstract

Abstract: An important prerequisite when annotating the remote sensing images by machine learning is that there are enough training samples for training, but labeling the samples is very time-consuming. In this paper, we solve the problem of unsupervised learning with small sample size in remote sensing image scene classification by domain adaptation method. A new domain adaptation framework is proposed which combines adversarial network and auxiliary task. Firstly, a novel remote sensing scene classification framework is established based on deep convolution neural networks. Secondly, a domain classifier is added to the network, in order to learn the domain-invariant features. The gradient direction of the domain loss is opposite to the label loss during the back propagation, which makes the domain predictor failed to distinguish the sample's domain. Lastly, we introduce an auxiliary task for the network, which augments the training samples and improves the generalization ability of the network. The experiments demonstrate better results in unsupervised classification with small sample sizes of remote sensing images compared to the baseline unsupervised domain adaptation approaches.

Key words: remote sensing image, scene classification, domain adaptation, deep convolutional neural network, adversarial network, multi-task learning

CLC Number:

P237

XU Suhui, MU Xiaodong, ZHANG Xiongmei, CHAI Dong. Unsupervised Remote Sensing Domain Adaptation Method with Adversarial Network and Auxiliary Task[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(12): 1969-1977.

References

[1] CHENG Gong,HAN Junwei,LU Xiaoqiang.Remote Sensing Image Scene Classification:Benchmark and State of the Art[J].Proceedings of the IEEE,2017,105(10):1865-1883.
[2] HU Fan,XIA Guisong,HU Jingwen,et al.Transferring Deep Convolutional Neural Networks for the Scene Classification of High-resolution Remote Sensing Imagery[J].Remote Sensing,2015,7(11):14680-14707.
[3] ZOU Qin,NI Lihao,ZHANG Tong,et al.Deep Learning Based Feature Selection for Remote Sensing Scene Classification[J].IEEE Geoscience and Remote Sensing Letters,2015,12(11):2321-2325.
[4] 许夙晖,慕晓冬,赵鹏,等.利用多尺度特征与深度网络对遥感影像进行场景分类[J].测绘学报,2016,45(7):834-840.DOI:10.11947/j.AGCS.2016.20150623. XU Suhui,MU Xiaodong,ZHAO Peng,et,al.Scene Classification of Remote Sensing Image Based on Multi-scale Feature and Deep Neural Network[J] Acta Geodaetica et Cartographica Sinica,2016,45(7):834-840.DOI:10.11947/j.AGCS.2016.20150623.
[5] ZHANG Fan,DU Bo,ZHANG Liangpei.Scene Classification via a Gradient Boosting Random Convolutional Network Framework[J].IEEE Transactions on Geoscience and Remote Sensing,2016,54(3):1793-1802.
[6] ROMERO A,GATTA C,CAMPS-VALLS G.Unsupervised Deep Feature Extraction for Remote Sensing Image Classification[J].IEEE Transactions on Geoscience and Remote Sensing,2016,54(3):1349-1362.
[7] NOGUEIRA K,PENATTI O A B,DOS SANTOS J A.Towards Better Exploiting Convolutional Neural Networks for Remote Sensing Scene Classification[J].Pattern Recognition,2017(61):539-556.
[8] ZHOU Xichuan,LI Shengli,TANG Fang,et al.Deep Learning with Grouped Features for Spatial Spectral Classification of Hyperspectral Images[J].IEEE Geoscience and Remote Sensing Letters,2017,14(1):97-101.
[9] GE Weifeng,YU Yizhou.Borrowing Treasures from the Wealthy:Deep Transfer Learning through Selective Joint Fine-tuning[J].arXiv preprint arXiv:1702.08690,2017.
[10] LIN Daoyu,FU Kun,WANG Yang,et al.MARTA GANs:Unsupervised Representation Learning for Remote Sensing Image Classification[J].IEEE Geoscience and Remote Sensing letters, 2017,14(11):2092-2096.
[11] 何小飞,邹峥嵘,陶超,等.联合显著性和多层卷积神经网络的高分影像场景分类[J].测绘学报,2016,45(9):1073-1080.DOI:10.11947/j.AGCS.2016.20150612. HE Xiaofei,ZOU Zhengrong,TAO Chao,et al.Combined Saliency with Multi-convolutional Neural Network for High Resolution Remote Sensing Scene Classification[J].Acta Geodaetica et Cartographica Sinica,2016,45(9):1073-1080.DOI:10.11947/j.AGCS.2016.20150612.
[12] WEISS K,KHOSHGOFTAAR T M,WANG D D.A Survey of Transfer Learning[J].Journal of Big Data,2016,3(1):9-48.
[13] KULIS B,SAENKO K,DARRELL T.What You Saw is not What You Get:Domain Adaptation Using Asymmetric Kernel Transforms[C]//Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Colorado Springs,CO,USA:IEEE,2011:1785-1792.
[14] HOFFMAN J,RODNER E,DONAHUE J,et al.Efficient Learning of Domain-invariant Image Representations[C]//Proceedings of the International Conference on Learning Representations (ICLR).Scottsdale,Arizona:ICLR,2013.
[15] DUAN Lixin,XU Dong,TSANG I W.Learning with Augmented Features for Heterogeneous Domain Adaptation[C]//Proceedings of the 29th International Conference on Machine Learning.Edinburgh,Scotland,UK:ICML,2012:711-718.
[16] GONG Boqing,SHI Yuan,SHA Fei,et al.Geodesic Flow Kernel for Unsupervised Domain Adaptation[C]//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition.Providence,RI,USA:IEEE,2012:2066-2073.
[17] GONG Boqing,GRAUMAN K,SHA Fei.Connecting the Dots with Landmarks:Discriminatively Learning Domain-invariant Features for Unsupervised Domain Adaptation[C]//Proceedings of the 30th International Conference on Machine Learning.Atlanta,USA:ICML,2013:222-230.
[18] TZENG E,HOFFMAN J,ZHANG Ning,et al.Deep Domain Confusion:Maximizing for Domain Invariance[J].arXiv preprint arXiv:1412.3474,2014.
[19] TZENG E,HOFFMAN J,DARRELL T,et al.Simultaneous Deep Transfer Across Domains and Tasks[C]//Proceedings of 2015 IEEE International Conference on Computer Vision.Santiago,Chile:IEEE,2015:4068-4076.
[20] GANIN Y,LEMPITSKY V.Unsupervised Domain Adaptation by Backpropagation[C]//Proceedings of the 32nd International Conference on Machine Learning.Lille,France:ICML,2015:1180-1189.
[21] GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Generative Adversarial Nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems.Montreal,Canada:MIT Press,2014:2672-2680.
[22] TUIA D,PERSELLO C,BRUZZONE L.Domain Adaptation for the Classification of Remote Sensing Data:An Overview of Recent Advances[J].IEEE Geoscience and Remote Sensing Magazine,2016,4(2):41-57.
[23] LI Xue,ZHANG Liangpei,DU Bo,et al.Iterative Reweighting Heterogeneous Transfer Learning Framework for Supervised Remote Sensing Image Classification[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2017,10(5):2022-2035.
[24] ZHANG Zhanpeng,LUO Ping,LOY C C,et al.Facial Landmark Detection by Deep Multi-task Learning[C]//Proceedings of the 13th European Conference on Computer Vision.Zurich,Switzerland:Springer,2014:94-108.
[25] SELTZER M L,DROPPO J.Multi-task Learning in Deep Neural Networks for Improved Phoneme Recognition[C]//Proceedings of 2013 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).Vancouver,BC,Canada:IEEE,2013:6965-6969.
[26] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenet Classification with Deep Convolutional Neural Networks[C]//Advances in Neural Information Processing Systems.Spain:NIPS,2012:1097-1105.
[27] VAN DER MAATEN L,HINTON G.Visualizing Data Using t-SNE[J].Journal of Machine Learning Research,2008,9(11):2579-2605.