Multi-scale entropy neural architecture search for object detection in remote sensing images

doi:10.11947/j.AGCS.2024.20230455

Abstract

Abstract:

Aiming at the traditional neural architecture search requires an enormous amount of time for supernet training, search efficiency is suboptimal, and the searched model can not efficiently solve the problem of multi-scale object detection difficulty and high background complexity in remote sensing images. This paper proposes a multi-scale entropy neural architecture search method for object detection in remote sensing images. At first, the feature separation convolution is added to the base block of the search space instead of the regular convolution in the residual block, which reduces the information redundancy in remote sensing images due to the high background complexity, and improves the detection performance of the network modeluner the complex background. Next, the maximum entropy principle is introduced to calculate the multi-scale entropy of each candidate network in the search space, and the multi-scale entropy is combined with the feature pyramid network to balance the detection of large, medium and small objects in remote sensing images. Finally, the network model with maximum multi-scale entropy is obtained by searching without parameter training using progressive evolutionary algorithm for the object detection task. The model ensures detection accuracy while improving the search efficiency. The proposed algorithm achieves a mean average precision of 93.1%, 75.5% and 73.6% on the RSOD, DIOR and DOTA datasets, respectively, with a network search time of 8.1 hours. The experimental results demonstrate that the proposed algorithm can significantly improve the search efficiency of the network, combine the features at different scales better and solve the problem of high image background complexity in the object detection task compared with the current benchmark methods.

Key words: remote sensing, neural architecture search, object detection, feature separation convolution, maximum entropy, multi-scale entropy, progressive evolution

CLC Number:

P237

Jun YANG, Hengjing XIE, Hongchao FAN, Haowen YAN. Multi-scale entropy neural architecture search for object detection in remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(7): 1384-1400.

Figures/Tables 22

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Tab.2

Tab.4

Tab.8

Fig.9

Tab.11

Fig.11

References 34

[1]	周鹏, 杨军. 采用神经网络架构搜索的遥感影像分割方法[J]. 西安电子科技大学学报, 2021, 48(5):47-57.
	ZHOU Peng, YANG Jun. Semantic segmentation of remote sensing images based on neural architecture search[J]. Journal of Xidian University, 2021, 48(5):47-57.
[2]	陈丁, 万刚, 李科. 多层特征与上下文信息相结合的光学遥感影像目标检测[J]. 测绘学报, 2019, 48(10):1275-1284. DOI: 10.11947/j.AGCS.2019.20180431.
	CHEN Ding, WAN Gang, LI Ke. Object detection in optical remote sensing images based on combination of multi-layer feature and context information[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(10):1275-1284. DOI: 10.11947/j.AGCS.2019.20180431.
[3]	CHEN Zhanlong, LI Shuangjiang, XU Yongyang, et al. Correg-YOLOv3: a method for dense buildings detection in high-resolution remote sensing images[J]. Journal of Geodesy and Geoinformation Science, 2023, 6(2):51-61. DOI: 10.11947/j.JGGS.2023.0206.
[4]	杨军, 韩鹏飞. 采用神经网络架构搜索的高分遥感影像目标检测 [J/OL]. 吉林大学学报 (工学版): 1-12 [2023-07-01]. DOI: 10.13229/j.cnki.jdxbgxb20221472.
	YANG Jun, HAN Pengfei. Object detection of high-resolution remote sensing images by neural architecture search [J/OL]. Journal of Jilin University (Engineering Edition): 1-12 [2023-07-01]. DOI: 10.13229/j.cnki.jdxbgxb20221472.
[5]	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE, 2014: 580-587.
[6]	ZOPH B, LE Q V. Neural architecture search with reinforcement learning [EB/OL]. [2023-08-01]. https://openreview.net/pdf?id=r1Ue8Hcxg.
[7]	PHAM H, GUAN M, ZOPH B, et al. Efficient neural architecture search via parameters sharing [C]//Proceedings of 2018 International Conference on Machine Learning. Stockholm: Springer, 2018: 4095-4104.
[8]	REAL E, AGGARWAL A, HUANG Y, et al. Regularized evolution for image classifier architecture search [C]//Proceedings of 2019 AAAI Conference on Artificial Intelligence. Hawaii: AAAI, 2019: 4780-4789.
[9]	LIU H, SIMONYAN K, YANG Y. DARTS: differentiable architecture search [EB/OL]. [2023-08-01]. https://arxiv.org/pdf/1806.09055v2.
[10]	GHIASI G, LIN T Y, LE Q V. NAS-FPN: learning scalable feature pyramid architecture for object detection[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 7036-7045.
[11]	LIANG Tingting, WANG Yongtao, TANG Zhi, et al. OPANAS: one-shot path aggregation network architecture search for object detection[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 10195-10203.
[12]	WANG Ning, GAO Yang, CHEN Hao, et al. NAS-FCOS: fast neural architecture search for object detection[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11940-11948.
[13]	CHEN Yukang, YANG Tong, ZHANG Xiangyu, et al. DetNAS: backbone search for object detection[J]. Advances in Neural Information Processing Systems, 2019, 32:6638-6648.
[14]	PENG Cheng, LI Yangyang, SHANG Ronghua, et al. RSBNet: one-shot neural architecture search for a backbone network in remote sensing image recognition[J]. Neurocomputing, 2023, 537:110-127.
[15]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[16]	WU Yuxin, HE Kaiming. Group normalization [C]//Proceedings of 2018 European Conference on Computer Vision. Cham: Springer, 2018: 3-19.
[17]	CHO K, VAN MERRIENBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2014: 1724-1734.
[18]	WANG Qilong, WU Banggu, ZHU Pengfei, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11534-11542.
[19]	HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
[20]	KESAVAN H K, KAPUR J N. The generalized maximum entropy principle[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1989, 19(5):1042-1052.
[21]	NIELSEN F, NOCK R. MaxEnt upper bounds for the differential entropy of univariate continuous distributions[J]. IEEE Signal Processing Letters, 2017, 24(4):402-406.
[22]	CHEN Qiang, WANG Yingming, YANG Tong, et al. You only look one-level feature[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 13039-13048.
[23]	NOH H, HONG S, HAN B. Learning deconvolution network for semantic segmentation[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1520-1528.
[24]	REDMON J, FARHADI A. YOLOv3: an incremental improvement[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 15:1125-1131.
[25]	TIAN Zhi, SHEN Chunhua, CHEN Hao, et al. FCOS: fully convolutional one-stage object detection[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 9627-9636.
[26]	LI Xiang, WANG Wenhai, WU Lijun, et al. Generalized focal loss: learning qualified and distributed bounding boxes for dense object detection [J]. Advances in Neural Information Processing Systems, 2020, 33:21002-21012.
[27]	LONG Yang, GONG Yiping, XIAO Zhifeng, et al. Accurate object localization in remote sensing images based on convolutional neural networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(5):2486-2498.
[28]	LI Ke, WAN Gang, CHENG Gong, et al. Object detection in optical remote sensing images: a survey and a new benchmark[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159:296-307.
[29]	XIA Guisong, BAI Xiang, DING Jian, et al. DOTA: a large-scale dataset for object detection in aerial images[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3974-3983.
[30]	XIAO Bin, TANG Han, JIANG Yanjun, et al. Brightness and contrast controllable image enhancement based on histogram specification[J]. Neurocomputing, 2018, 275:2798-2809.
[31]	HE Zhezhi, RAKIN A S, FAN Deliang. Parametric noise injection: trainable randomness to improve deep neural network robustness against adversarial attack[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 588-597.
[32]	REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149.
[33]	WIGHTMAN R, TOUVRON H, JÉGOU H. ResNet strikes back: an improved training procedure in timm [EB/OL]. [2023-08-01]. https://openreview.net/pdf?id=NG6MJnVl6M5.
[34]	ZHANG Haoyang, WANG Ying, DAYOUB F, et al. VarifocalNet: an IoU-aware dense object detector[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 8514-8523.

不同数据集的预处理	浮点运算数	参数量	mAP/（%）	AP₅₀/（%）	AP₇₅/（%）	AP_s/（%）	AP_m/（%）	AP_l/（%）
未经过数据预处理（RSOD）	207.63×10⁹	30.29×10⁶	92.4	99.2	98.5	67.6	89.8	94.7
经过数据预处理（RSOD）	207.63×10⁹	30.29×10⁶	93.1	99.2	98.7	68.6	89.9	95.4
未经过数据预处理（DIOR）	144.41×10⁹	30.04×10⁶	72.1	90.8	80.7	38.5	68.0	84.8
经过数据预处理（DIOR）	144.41×10⁹	30.04×10⁶	75.5	92.6	84.4	44.3	72.1	87.0
未经过数据预处理（DOTA）	208.15×10⁹	30.31×10⁶	70.8	87.5	81.6	50.0	76.6	82.5
经过数据预处理（DOTA）	208.15×10⁹	30.31×10⁶	73.6	89.2	83.9	53.4	80.5	85.0

模块	卷积核尺寸	输入通道	输出通道	步长	瓶颈层	层数
Conv	3	3	32	2	—	1
FSResBlock	5	32	128	2	40	3
FSResBlock	5	128	512	2	64	7
FSResBlock	5	512	1200	2	128	7
FSResBlock	5	1200	1200	1	136	7
FSResBlock	3	1200	2048	2	512	1

网络模型	浮点运算数	参数量	网络搜索时间/h	mAP/（%）	AP₅₀/（%）	AP₇₅/（%）	AP_s/（%）	AP_m/（%）	AP_l/（%）
Faster R-CNN	841.40×10⁹	41.14×10⁶	—	84.9	99.5	96.4	61.0	83.0	88.0
ResNet-SB	298.57×10⁹	46.89×10⁶	—	77.4	97.5	90.5	23.3	66.4	82.3
VarifocalNet	218.57×10⁹	32.49×10⁶	—	87.8	99.1	96.9	56.0	82.9	91.3
NAS-FCOS	216.31×10⁹	38.67×10⁶	11.4	77.3	98.0	89.2	26.9	75.0	83.1
DetNAS	201.67×10⁹	32.79×10⁶	12.6	85.5	99.5	97.9	63.2	83.2	88.6
本文方法	207.63×10⁹	30.29×10⁶	8.1	93.1	99.2	98.7	68.6	89.9	95.4

网络模型	飞机	操场	立交桥	储油罐
Faster R-CNN	78.7	92.2	81.2	87.6
ResNet-SB	60.3	93.1	81.4	74.8
VarifocalNet	77.1	96.2	89.8	88.2
NAS-FCOS	66.3	86.8	72.6	83.5
DetNAS	79.9	90.0	85.2	87.9
本文方法	84.5	98.7	96.2	93.3

网络模型	浮点运算数	参数量	mAP/（%）	AP₅₀/（%）	AP₇₅/（%）	AP_s/（%）	AP_m/（%）	AP_l/（%）
Faster R-CNN	798.04×10⁹	32.95×10⁶	66.1	88.8	74.2	12.2	50.8	76.2
ResNet-SB	216.40×10⁹	41.22×10⁶	52.0	76.4	57.5	9.7	31.2	62.1
VarifocalNet	159.35×10⁹	32.49×10⁶	70.1	89.6	78.3	35.2	64.6	83.0
NAS-FCOS	230.62×10⁹	38.15×10⁶	55.7	80.4	60.6	10.4	35.8	65.5
DetNAS	208.47×10⁹	25.38×10⁶	67.8	91.3	78.3	36.1	60.6	74.4
本文方法	144.41×10⁹	30.04×10⁶	75.5	92.6	84.4	44.3	72.1	87.0