面向地理路网的交通信号智能协同控制方法

doi:10.11947/j.AGCS.2021.20210191

测绘学报 ›› 2021, Vol. 50 ›› Issue (9): 1203-1210.doi: 10.11947/j.AGCS.2021.20210191

面向地理路网的交通信号智能协同控制方法

郑晔^1,2, 郭仁忠^1,2, 马丁^1,2, 赵志刚^1,2, 李晓明^1,2

1. 深圳大学建筑与城市规划学院智慧城市研究院, 广东深圳 518060;
2. 深圳市空间信息智能感知与服务重点实验室, 广东深圳 518060

收稿日期:2021-04-13 修回日期:2021-06-21 发布日期:2021-10-09
通讯作者: 赵志刚 E-mail:zhaozgrisc@szu.edu.cn
作者简介:郑晔(1989-),男,博士,博士后,研究方向为地理AI。E-mail:zhengye@szu.edu.cn
基金资助:
国家重点研发计划（2018YFB2100700；2019YFB2103104；2019YFB210310）；中国博士后基金（2019M663070）

Multi-agent cooperative control for traffic signal on geographic road network

ZHENG Ye^1,2, GUO Renzhong^1,2, MA Ding^1,2, ZHAO Zhigang^1,2, LI Xiaoming^1,2

1. Research Institute for Smart Cities, School of Architecture and Urban Planning, Shenzhen University, Shenzhen 518060, China;
2. Key Laboratory of Spatial Information Smart Sensing and Services, Shenzhen 518060, China

Received:2021-04-13 Revised:2021-06-21 Published:2021-10-09
Supported by:
The National Key Research and Development Program of China (Nos. 2018YFB2100700; 2019YFB2103104; 2019YFB210310); China Postdoctoral Science Foundation (No. 2019M663070)

摘要/Abstract

摘要： 城市交通运行效率是影响城市生产力发展的重要因素之一，也是智慧城市建设过程中的重要研究课题。随着计算机技术的发展，人工智能特别是强化学习在交通信号控制中发挥重要作用。目前，基于强化学习的交通信号控制主要针对单路口或城市干道进行优化，面向城市地理路网区域协调控制研究较少。本文结合马尔可夫序列决策，提出一种基于强化学习的双层智能体协同控制方法。第1层，针对单个路口实现粗调训练，智能体通过观察路口每一车道的排队长度调控信号配时，实现单个路口不堵塞；第2层，将多个粗调训练后的智能体模型放入地理网络中，实现多路口的协同微调训练。本文以宁波某中学片区的交通协调为优化目标展开试验。结果表明，调控方法与原有固定配时方案相比，具有更高的通行效率。

关键词: 地理路网, 交通信号控制, 协同控制, 强化学习

Abstract: Urban traffic efficiency is one of the key factors affecting urban productivity and is also a crucial topic in the process of smart city construction. With the development of computer technology, artificial intelligence, especially reinforcement learning, plays an increasingly important role in traffic signal control. Currently, traffic signal control based on reinforcement learning is mainly used for the optimization for simple scenarios, such as single road intersection or urban arterial road, not yet for regional coordinated control on an urban geographic road network. This paper is motivated to fill this gap by proposing a two-layered agent cooperative control approach based on reinforcement learning. The first layer implements a coarse-tuning training at a single intersection, where the agents make the single intersection non-blocking by observing the queue length for each lane; In the second layer, the coarse-tuning-trained agent models are put into the geographic network to execute the cooperative fine-tuning training at multiple intersections. This paper conducts the optimization-orientated traffic coordination through a case study of a middle school area in Ningbo. The results show that our control approach is superior to the traditional fixed timing scheme in terms of the passage efficiency.

Key words: geographic road network, traffic signal control, cooperative control, reinforcement learning

中图分类号:

P208

郑晔, 郭仁忠, 马丁, 赵志刚, 李晓明. 面向地理路网的交通信号智能协同控制方法[J]. 测绘学报, 2021, 50(9): 1203-1210.

ZHENG Ye, GUO Renzhong, MA Ding, ZHAO Zhigang, LI Xiaoming. Multi-agent cooperative control for traffic signal on geographic road network[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(9): 1203-1210.

参考文献

[1] 郭仁忠, 林浩嘉, 贺彪, 等. 面向智慧城市的GIS框架[J]. 武汉大学学报(信息科学版), 2020, 45(12): 1829-1835. GUO Renzhong, LIN Haojia, HE Biao, et al. GIS framework for smart cities[J]. Geomatics and Information Science of Wuhan University, 2020, 45(12): 1829-1835.
[2] MORGAN J T, LITTLE J D C. Synchronizing traffic signals for maximal bandwidth[J]. Operations Research, 1964, 12(6): 896-912.
[3] LITTLE J D C. The synchronization of traffic signals by mixed-integer linear programming[J]. Operations Research, 1966, 14(4): 568-594.
[4] GARTNER N H, ASSMAN S F, LASAGA F, et al. A multi-band approach to arterial traffic signal optimization[J]. Transportation Research Part B: Methodological, 1991, 25(1): 55-74.
[5] PILLAI R S, RATHI| A K, L COHEN S. A restricted branch-and-bound approach for generating maximum bandwidth signal timing plans for traffic networks[J]. Transportation Research Part B: Methodological, 1998, 32(8): 517-529.
[6] PAPPIS C P, MAMDANI E H. A fuzzy logic controller for a traffic junction[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1977, 7(10): 707-717.
[7] LI Jinjian, DRIDI M, EL-MOUDNI A. A cooperative traffic control for the vehicles in the intersection based on the genetic algorithm[C]//2016 4th IEEE International Colloquium on Information Science and Technology (CiSt). Tangier, Morocco:IEEE, 2016: 627-632.
[8] HOSSAIN M S, SINHA H, MUSTAFA R. A Belief rule based expert system to control traffic signals under uncertainty[C]//Proceedings of 2015 International Conference on Computer and Information Engineering (ICCIE). Rajshahi, Bangladesh: IEEE, 2015: 83-86.
[9] 孙文彬, 熊婷. 历史数据和强化学习相结合的低频轨迹数据匹配算法[J]. 测绘学报, 2016, 45(11): 1328-1334. SUN Wenbin, XIONG Ting. A low-sampling-rate trajectory matching algorithm in combination of history trajectory and reinforcement learning[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(11): 1328-1334.
[10] ABDOOS M, MOZAYANI N, BAZZAN A L C. Traffic light control in non-stationary environments based on multi agent Q-learning[C]//Proceedings of 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC). Washington, DC, USA: IEEE, 2011: 1580-1585.
[11] CHIN Y K, LEE L K, BOLONG N, et al. Exploring Q-learning optimization in traffic signal timing plan management[C]//Proceedings of 2011 Third International Conference on Computational Intelligence, Communication Systems and Networks. Bali, Indonesia: IEEE, 2011: 269-274.
[12] BRYS T, PHAM T T, TAYLOR M E. Distributed learning and multi-objectivity in traffic light control[J]. Connection Science, 2014, 26(1): 65-83.
[13] NING Zhaolong, ZHANG Kaiyuan, WANG Xiaojie, et al. Joint computing and caching in 5G-envisioned Internet of vehicles: a deep reinforcement learning-based traffic control system[J]. IEEE Transactions on Intelligent Transportation Systems, 2020,0276(99): 1-12.
[14] ZHOU Pengyuan, BRAUD T, ALHILAL A, et al. ERL: edge based reinforcement learning for optimized urban traffic light control[C]//Proceedings of 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). Kyoto, Japan: IEEE, 2019: 849-854.
[15] JOO H, AHMED S H, LIM Y. Traffic signal control for smart cities using reinforcement learning[J]. Computer Communications, 2020, 154: 324-330.
[16] LIANG Xiaoyuan, DU Xunsheng, WANG Guiling, et al. A deep reinforcement learning network for traffic light cycle control[J]. IEEE Transactions on Vehicular Technology, 2019, 68(2): 1243-1253.
[17] ZHENG Ye, GUO Renzhong, MA Ding, et al. A novel approach to coordinating green wave system with adaptation evolutionary strategy[J]. IEEE Access, 2020, 8: 214115-214127.
[18] GAO Yong, JIANG Dan, XU Yan. Optimize taxi driving strategies based on reinforcement learning[J]. International Journal of Geographical Information Science, 2018, 32(8): 1677-1696.
[19] HU Yujing, DA Qing, ZENG Anxiang, et al. Reinforcement learning to rank in E-commerce search engine: formalization, analysis, and application[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, NY, USA: ACM, 2018: 368-377.
[20] SUTTON R, BARTO A. Reinforcement learning: an introduction [M]: MIT: MIT Press, 1998.
[21] LITTMAN M L. Reinforcement learning improves behaviour from evaluative feedback[J]. Nature, 2015, 521(7553): 445-451.
[22] KRAJZEWICZ D, HERTKORN G, RÖSSEL C, et al. SUMO (simulation of urban mobility); an open-source traffic simulation [C]//Proceedings of the 4th Middle East Symposium on Simulation and Modelling (MESM2002). American University, Sharjah, United Arab Emirates: MESM,2002.
[23] WEBSTER F V. Traffic signal settings[R]. London:Road Research Laboratory, 1958.

面向地理路网的交通信号智能协同控制方法

Multi-agent cooperative control for traffic signal on geographic road network

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 2

编辑推荐

Metrics

本文评价

[1]	叶浩宇, 涂伟, 叶贺辉, 麦可, 赵天鸿, 李清泉. 基于深度强化学习的电动出租车运营优化[J]. 测绘学报, 2020, 49(12): 1630-1639.
[2]	孙文彬, 熊婷. 历史数据和强化学习相结合的低频轨迹数据匹配算法[J]. 测绘学报, 2016, 45(11): 1328-1334.