Multi-scale fully convolutional neural network for building extraction

doi:10.11947/j.AGCS.2019.20180062

Abstract

Abstract: Some holes occurred when extracting large buildings in high spatial resolution remote sensing images with VGG16. A method of building extraction based on multi-scale features is proposed to solve this problem. Firstly, the original images were downsampled at different scales. Then, it could be extracted that the features of buildings at different scales and fused them. To reduce the number of network parameters, the fully convolutional upsampling was used to replace the fully connected layer in the original VGG16 model. The study images were from the 0.5 m resolution in Jading of Shanghai and 1 m resolution Massachusetts building dataset. The accuracy of buildings extraction were 97.09% and 96.66% respectively. The result showed the effectiveness of the proposed method.

Key words: large buildings, multi-scale, fully convolutional upsampling

CLC Number:

TP79
P237

CUI Weihong, XIONG Baoyu, ZHANG Liyao. Multi-scale fully convolutional neural network for building extraction[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(5): 597-608.

References

[1] JIN Xiaoying, DAVIS C H. Automated building extraction from high-resolution satellite imagery in urban areas using structural, contextual, and spectral information[J]. EURASIP Journal on Advances in Signal Processing, 2005(2005):745309.
[2] PESARESI M, GERHARDINGER A, KAYITAKIRE F. A robust built-up area presence index by anisotropic rotation-invariant textural measure[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2008, 1(3):180-192.
[3] SIRMACEK B, UNSALAN C. A probabilistic framework to detect buildings in aerial and satellite images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(1):211-221.
[4] HARRIS C, STEPHENS M. A combined corner and edge detector[C]//Proceedings of the 4th Alvey Vision Conference. Manchester:BMVA, 1988:147-151.
[5] UNSALAN C. Gradient-magnitude-based support regions in structural land use classification[J]. IEEE Geoscience and Remote Sensing Letters, 2006, 3(4):546-550.
[6] WANG Jun, YANG Xiucheng, QIN Xuebin, et al. An efficient approach for automatic rectangular building extraction from very high resolution optical satellite imagery[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(3):487-491.
[7] AKINLAR C, TOPAL C. EDLines:A real-time line segment detector with a false detection control[J]. Pattern Recognition Letters, 2011, 32(13):1633-1642.
[8] 吕凤华, 舒宁, 龚龑, 等. 利用多特征进行航空影像建筑物提取[J]. 武汉大学学报(信息科学版), 2017, 42(5):656-660. LÜ Fenghua, SHU Ning, GONG Yan, et al. Regular building extraction from high resolution image based on multilevel-features[J]. Geomatics and Information Science of Wuhan University, 2017, 42(5):656-660.
[9] HUANG Xin, ZHANG Liangpei. A multidirectional and multiscale morphological index for automatic building extraction from multispectral GeoEye-1 imagery[J]. Photogrammetric Engineering & Remote Sensing, 2011, 77(7):721-732.
[10] 胡荣明, 黄小兵, 黄远程. 增强形态学建筑物指数应用于高分辨率遥感影像中建筑物提取[J]. 测绘学报, 2014, 43(5):514-520. DOI:10.13485/j.cnki.11-2089.2014.0084. HU Rongming, HUANG Xiaobing, HUANG Yuancheng. An enhanced morphological building index for building extraction from high-resolution image[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(5):514-520. DOI:10.13485/j.cnki.11-2089.2014.0084.
[11] 林祥国, 张继贤. 面向对象的形态学建筑物指数及其高分辨率遥感影像建筑物提取应用[J]. 测绘学报, 2017, 46(6):724-733. DOI:10.11947/j.AGCS.2017.20170068. LIN Xiangguo, ZHANG Jixian. Object-based morphological building index for building extraction from high resolution remote sensing imagery[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(6):724-733. DOI:10.11947/j.AGCS.2017.20170068.
[12] MNIH V. Machine learning for aerial image labeling[D]. Toronto:University of Toronto, 2013.
[13] SAITO S, YAMASHITA T, AOKI Y. Multiple object extraction from aerial imagery with convolutional neural networks[J]. Journal of Imaging Science and Technology, 2016, 60(1):10402-1-10402-9.
[14] MAGGIORI E, TARABALKA Y, CHARPIAT G, et al. Convolutional neural networks for large-scale remote-sensing image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(2):645-657.
[15] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA:IEEE, 2015:3431-3440.
[16] MARCU A, LEORDEANU M. Dual local-global contextual pathways for recognition in aerial imagery[R/OL]. (2016-05-12)[2017-08-13]. https://sciencewise.info/articles/1605.05462.
[17] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada:ACM, 2012:1097-1105.
[18] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]//Proceedings of International Conference on Learning Representations 2015. San Diego:ICLR, 2015:463-476.
[19] LIU Wei, ANGUELOV D, ERHAN D, et al. SSD:single shot multi box detector[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam:Springer, 2016:21-37.
[20] ZHAO Hengshuang, SHI Jianping, QI Xiaojuan, et al. Pyramid scene parsing network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI:IEEE, 2017:6230-6239.
[21] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016:770-778.
[22] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab:semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):834-848.
[23] FARABET C, COUPRIE C, NAJMAN L, et al. Learning hierarchical features for scene labeling[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(8):1915-1929.
[24] LIN Guosheng, SHEN Chunhua, VAN DEN HENGEL A, et al. Efficient piecewise training of deep structured models for semantic segmentation[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016:3194-3203.
[25] CHEN L C, YANG Yi, WANG Jiang, et al. Attention to scale:scale-aware semantic image segmentation[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016:3640-3649.
[26] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet:a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2491-2495.
[27] DENG Jia, DONG Wei, SOCHER R, et al. ImageNet:a large-scale hierarchical image database[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Miami Beach, FL:IEEE, 2009:248-255.
[28] BISHOP C M. Neural networks for pattern recognition[M]. Oxford:Oxford University Press, 1995.