遥感影像目标检测多尺度熵神经网络架构搜索

doi:10.11947/j.AGCS.2024.20230455

摘要/Abstract

摘要：

针对传统神经网络架构搜索需要耗费大量时间用于超网训练，搜索效率较低，搜索得到的模型无法高效解决遥感影像中多尺度目标检测困难、背景复杂度高的问题，本文提出采用多尺度熵神经网络架构搜索的方法进行遥感影像目标检测。首先，在搜索空间的基础模块中加入特征分离卷积以代替残差模块中的常规卷积，减少遥感影像中由于背景复杂度高而造成的信息间干扰，提高网络模型在复杂背景下的检测性能；然后，引入最大熵原理，计算搜索空间中每个候选网络的多尺度熵，将多尺度熵与特征金字塔网络相结合，以兼顾遥感影像大、中、小目标的检测；最后，在不进行参数训练的情况下利用渐进式进化算法搜索得到多尺度熵最大的网络模型用于目标检测任务，在保证模型检测精度的同时，提升网络搜索效率。本文方法在RSOD、DIOR和DOTA数据集上的平均检测精度均值分别达到93.1%、75.5%和73.6%，网络搜索时间为8.1 h。试验结果表明，与当前基准方法相比，本文方法能够显著提升网络的搜索效率，在目标检测任务中更好地结合了不同尺度下的特征并解决了影像背景复杂度高的问题。

关键词: 遥感影像, 神经网络架构搜索, 目标检测, 特征分离卷积, 最大熵, 多尺度熵, 渐进式进化

Abstract:

Aiming at the traditional neural architecture search requires an enormous amount of time for supernet training, search efficiency is suboptimal, and the searched model can not efficiently solve the problem of multi-scale object detection difficulty and high background complexity in remote sensing images. This paper proposes a multi-scale entropy neural architecture search method for object detection in remote sensing images. At first, the feature separation convolution is added to the base block of the search space instead of the regular convolution in the residual block, which reduces the information redundancy in remote sensing images due to the high background complexity, and improves the detection performance of the network modeluner the complex background. Next, the maximum entropy principle is introduced to calculate the multi-scale entropy of each candidate network in the search space, and the multi-scale entropy is combined with the feature pyramid network to balance the detection of large, medium and small objects in remote sensing images. Finally, the network model with maximum multi-scale entropy is obtained by searching without parameter training using progressive evolutionary algorithm for the object detection task. The model ensures detection accuracy while improving the search efficiency. The proposed algorithm achieves a mean average precision of 93.1%, 75.5% and 73.6% on the RSOD, DIOR and DOTA datasets, respectively, with a network search time of 8.1 hours. The experimental results demonstrate that the proposed algorithm can significantly improve the search efficiency of the network, combine the features at different scales better and solve the problem of high image background complexity in the object detection task compared with the current benchmark methods.

Key words: remote sensing, neural architecture search, object detection, feature separation convolution, maximum entropy, multi-scale entropy, progressive evolution

中图分类号:

P237

杨军, 解恒静, 范红超, 闫浩文. 遥感影像目标检测多尺度熵神经网络架构搜索[J]. 测绘学报, 2024, 53(7): 1384-1400.

Jun YANG, Hengjing XIE, Hongchao FAN, Haowen YAN. Multi-scale entropy neural architecture search for object detection in remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(7): 1384-1400.

图/表 22

图1

图2

图3

图4

图5

图6

表1

表2

图7

表3

表4

图8

表5

表6

表7

表8

图9

图10

表9

表10

表11

图11

参考文献 34

[1]	周鹏, 杨军. 采用神经网络架构搜索的遥感影像分割方法[J]. 西安电子科技大学学报, 2021, 48(5):47-57.
	ZHOU Peng, YANG Jun. Semantic segmentation of remote sensing images based on neural architecture search[J]. Journal of Xidian University, 2021, 48(5):47-57.
[2]	陈丁, 万刚, 李科. 多层特征与上下文信息相结合的光学遥感影像目标检测[J]. 测绘学报, 2019, 48(10):1275-1284. DOI: 10.11947/j.AGCS.2019.20180431.
	CHEN Ding, WAN Gang, LI Ke. Object detection in optical remote sensing images based on combination of multi-layer feature and context information[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(10):1275-1284. DOI: 10.11947/j.AGCS.2019.20180431.
[3]	CHEN Zhanlong, LI Shuangjiang, XU Yongyang, et al. Correg-YOLOv3: a method for dense buildings detection in high-resolution remote sensing images[J]. Journal of Geodesy and Geoinformation Science, 2023, 6(2):51-61. DOI: 10.11947/j.JGGS.2023.0206.
[4]	杨军, 韩鹏飞. 采用神经网络架构搜索的高分遥感影像目标检测 [J/OL]. 吉林大学学报 (工学版): 1-12 [2023-07-01]. DOI: 10.13229/j.cnki.jdxbgxb20221472.
	YANG Jun, HAN Pengfei. Object detection of high-resolution remote sensing images by neural architecture search [J/OL]. Journal of Jilin University (Engineering Edition): 1-12 [2023-07-01]. DOI: 10.13229/j.cnki.jdxbgxb20221472.
[5]	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE, 2014: 580-587.
[6]	ZOPH B, LE Q V. Neural architecture search with reinforcement learning [EB/OL]. [2023-08-01]. https://openreview.net/pdf?id=r1Ue8Hcxg.
[7]	PHAM H, GUAN M, ZOPH B, et al. Efficient neural architecture search via parameters sharing [C]//Proceedings of 2018 International Conference on Machine Learning. Stockholm: Springer, 2018: 4095-4104.
[8]	REAL E, AGGARWAL A, HUANG Y, et al. Regularized evolution for image classifier architecture search [C]//Proceedings of 2019 AAAI Conference on Artificial Intelligence. Hawaii: AAAI, 2019: 4780-4789.
[9]	LIU H, SIMONYAN K, YANG Y. DARTS: differentiable architecture search [EB/OL]. [2023-08-01]. https://arxiv.org/pdf/1806.09055v2.
[10]	GHIASI G, LIN T Y, LE Q V. NAS-FPN: learning scalable feature pyramid architecture for object detection[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 7036-7045.
[11]	LIANG Tingting, WANG Yongtao, TANG Zhi, et al. OPANAS: one-shot path aggregation network architecture search for object detection[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 10195-10203.
[12]	WANG Ning, GAO Yang, CHEN Hao, et al. NAS-FCOS: fast neural architecture search for object detection[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11940-11948.
[13]	CHEN Yukang, YANG Tong, ZHANG Xiangyu, et al. DetNAS: backbone search for object detection[J]. Advances in Neural Information Processing Systems, 2019, 32:6638-6648.
[14]	PENG Cheng, LI Yangyang, SHANG Ronghua, et al. RSBNet: one-shot neural architecture search for a backbone network in remote sensing image recognition[J]. Neurocomputing, 2023, 537:110-127.
[15]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[16]	WU Yuxin, HE Kaiming. Group normalization [C]//Proceedings of 2018 European Conference on Computer Vision. Cham: Springer, 2018: 3-19.
[17]	CHO K, VAN MERRIENBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2014: 1724-1734.
[18]	WANG Qilong, WU Banggu, ZHU Pengfei, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11534-11542.
[19]	HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
[20]	KESAVAN H K, KAPUR J N. The generalized maximum entropy principle[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1989, 19(5):1042-1052.
[21]	NIELSEN F, NOCK R. MaxEnt upper bounds for the differential entropy of univariate continuous distributions[J]. IEEE Signal Processing Letters, 2017, 24(4):402-406.
[22]	CHEN Qiang, WANG Yingming, YANG Tong, et al. You only look one-level feature[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 13039-13048.
[23]	NOH H, HONG S, HAN B. Learning deconvolution network for semantic segmentation[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1520-1528.
[24]	REDMON J, FARHADI A. YOLOv3: an incremental improvement[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 15:1125-1131.
[25]	TIAN Zhi, SHEN Chunhua, CHEN Hao, et al. FCOS: fully convolutional one-stage object detection[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 9627-9636.
[26]	LI Xiang, WANG Wenhai, WU Lijun, et al. Generalized focal loss: learning qualified and distributed bounding boxes for dense object detection [J]. Advances in Neural Information Processing Systems, 2020, 33:21002-21012.
[27]	LONG Yang, GONG Yiping, XIAO Zhifeng, et al. Accurate object localization in remote sensing images based on convolutional neural networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(5):2486-2498.
[28]	LI Ke, WAN Gang, CHENG Gong, et al. Object detection in optical remote sensing images: a survey and a new benchmark[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159:296-307.
[29]	XIA Guisong, BAI Xiang, DING Jian, et al. DOTA: a large-scale dataset for object detection in aerial images[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3974-3983.
[30]	XIAO Bin, TANG Han, JIANG Yanjun, et al. Brightness and contrast controllable image enhancement based on histogram specification[J]. Neurocomputing, 2018, 275:2798-2809.
[31]	HE Zhezhi, RAKIN A S, FAN Deliang. Parametric noise injection: trainable randomness to improve deep neural network robustness against adversarial attack[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 588-597.
[32]	REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149.
[33]	WIGHTMAN R, TOUVRON H, JÉGOU H. ResNet strikes back: an improved training procedure in timm [EB/OL]. [2023-08-01]. https://openreview.net/pdf?id=NG6MJnVl6M5.
[34]	ZHANG Haoyang, WANG Ying, DAYOUB F, et al. VarifocalNet: an IoU-aware dense object detector[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 8514-8523.

不同数据集的预处理	浮点运算数	参数量	mAP/（%）	AP₅₀/（%）	AP₇₅/（%）	AP_s/（%）	AP_m/（%）	AP_l/（%）
未经过数据预处理（RSOD）	207.63×10⁹	30.29×10⁶	92.4	99.2	98.5	67.6	89.8	94.7
经过数据预处理（RSOD）	207.63×10⁹	30.29×10⁶	93.1	99.2	98.7	68.6	89.9	95.4
未经过数据预处理（DIOR）	144.41×10⁹	30.04×10⁶	72.1	90.8	80.7	38.5	68.0	84.8
经过数据预处理（DIOR）	144.41×10⁹	30.04×10⁶	75.5	92.6	84.4	44.3	72.1	87.0
未经过数据预处理（DOTA）	208.15×10⁹	30.31×10⁶	70.8	87.5	81.6	50.0	76.6	82.5
经过数据预处理（DOTA）	208.15×10⁹	30.31×10⁶	73.6	89.2	83.9	53.4	80.5	85.0

模块	卷积核尺寸	输入通道	输出通道	步长	瓶颈层	层数
Conv	3	3	32	2	—	1
FSResBlock	5	32	128	2	40	3
FSResBlock	5	128	512	2	64	7
FSResBlock	5	512	1200	2	128	7
FSResBlock	5	1200	1200	1	136	7
FSResBlock	3	1200	2048	2	512	1

网络模型	浮点运算数	参数量	网络搜索时间/h	mAP/（%）	AP₅₀/（%）	AP₇₅/（%）	AP_s/（%）	AP_m/（%）	AP_l/（%）
Faster R-CNN	841.40×10⁹	41.14×10⁶	—	84.9	99.5	96.4	61.0	83.0	88.0
ResNet-SB	298.57×10⁹	46.89×10⁶	—	77.4	97.5	90.5	23.3	66.4	82.3
VarifocalNet	218.57×10⁹	32.49×10⁶	—	87.8	99.1	96.9	56.0	82.9	91.3
NAS-FCOS	216.31×10⁹	38.67×10⁶	11.4	77.3	98.0	89.2	26.9	75.0	83.1
DetNAS	201.67×10⁹	32.79×10⁶	12.6	85.5	99.5	97.9	63.2	83.2	88.6
本文方法	207.63×10⁹	30.29×10⁶	8.1	93.1	99.2	98.7	68.6	89.9	95.4

网络模型	飞机	操场	立交桥	储油罐
Faster R-CNN	78.7	92.2	81.2	87.6
ResNet-SB	60.3	93.1	81.4	74.8
VarifocalNet	77.1	96.2	89.8	88.2
NAS-FCOS	66.3	86.8	72.6	83.5
DetNAS	79.9	90.0	85.2	87.9
本文方法	84.5	98.7	96.2	93.3

网络模型	浮点运算数	参数量	mAP/（%）	AP₅₀/（%）	AP₇₅/（%）	AP_s/（%）	AP_m/（%）	AP_l/（%）
Faster R-CNN	798.04×10⁹	32.95×10⁶	66.1	88.8	74.2	12.2	50.8	76.2
ResNet-SB	216.40×10⁹	41.22×10⁶	52.0	76.4	57.5	9.7	31.2	62.1
VarifocalNet	159.35×10⁹	32.49×10⁶	70.1	89.6	78.3	35.2	64.6	83.0
NAS-FCOS	230.62×10⁹	38.15×10⁶	55.7	80.4	60.6	10.4	35.8	65.5
DetNAS	208.47×10⁹	25.38×10⁶	67.8	91.3	78.3	36.1	60.6	74.4
本文方法	144.41×10⁹	30.04×10⁶	75.5	92.6	84.4	44.3	72.1	87.0