基于深度强化学习的电动出租车运营优化

doi:10.11947/j.AGCS.2020.20190516

测绘学报 ›› 2020, Vol. 49 ›› Issue (12): 1630-1639.doi: 10.11947/j.AGCS.2020.20190516

基于深度强化学习的电动出租车运营优化

叶浩宇¹, 涂伟^2,3,4,5, 叶贺辉⁶, 麦可^2,3,4, 赵天鸿^2,4, 李清泉^1,2,3,4

1. 武汉大学测绘遥感信息工程国家重点实验室, 湖北武汉 430079;
2. 深圳大学建筑与城市规划学院城市空间信息工程系, 广东深圳 518060;
3. 人工智能与数字经济广东省实验室(深圳), 广东深圳 518060;
4. 广东省城市空间信息工程重点实验室, 广东深圳 518060;
5. 自然资源部大湾区地理环境监测重点实验室, 广东深圳 518060;
6. 闽江学院软件学院, 福建福州 350108

收稿日期:2019-12-16 修回日期:2020-06-07 发布日期:2020-12-25
通讯作者: 涂伟 E-mail:tuwei@szu.edu.cn
作者简介:叶浩宇(1996-),男,硕士生,研究方向为城市时空大数据分析。E-mail:yehaoyuchn@whu.edu.cn
基金资助:
国家重点研发计划（2019YFB2103104）；广东省自然科学基金（2019A1515011049）；深圳市科技创新委员会基础研究（JCYJ20170412105839839）

Deep reinforcement learning based electric taxi service optimization

YE Haoyu¹, TU Wei^2,3,4,5, YE Hehui⁶, MAI Ke^2,3,4, ZHAO Tianhong^2,4, LI Qingquan^1,2,3,4

1. State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan 430079, China;
2. Department of Urban Informatics, School of Architecture and Urban Planning, Shenzhen University, Shenzhen 518060, China;
3. Guangdong Laboratory of Artificial Intelligence and Digital Economy(SZ), Shenzhen University, Shenzhen 518060, China;
4. Guangdong Key Laboratory of Urban Informatics, Shenzhen University, Shenzhen 518060, China;
5. Key Laboratory for Geo-Environmental Monitoring of Great Bay Area, MNR, Shenzhen 518060, China;
6. Software College, Minjiang University, Fuzhou 350108, China

Received:2019-12-16 Revised:2020-06-07 Published:2020-12-25
Supported by:
The National Key Research and Development Program of China (No. 2019YFB2103104);The Natural Science Foundation of Guangdong Province(No. 2019A1515011049);The Basic Research Projects of Shenzhen Technology Innovation Commission (No. JCYJ20170412105839839)

摘要/Abstract

摘要： 作为公共交通的重要组成部分，电动出租车对电动车推广具有重要的示范意义。相较于燃油出租车，电动出租车需要耗费更多充电时间，降低了出租车司机的使用意愿，全面推广面临较大阻力。强化学习方法方兴未艾，适用于出租车运营的顺序决策过程。基于强化学习，本文构建双深度Q学习网络（double deep Q-learning network，DDQN）模型模拟电动出租车的运行。根据出租车的实时状态选择并执行最优载客、充电、空驶和等待等动作，通过训练得到全局最优的电动出租车运营策略，实现电动出租车运营智能优化。利用美国纽约市曼哈顿岛的出租车出行数据进行试验。结果表明：相较于简单的电动出租车运营模式，DDQN优化策略最高将充电等待时长降低70%，拒载率降低53%，司机的载客收入提高7%。相对于电池容量，充电速率和车辆总数对出租车运营收入影响更大，当充电速率达到120 kW时，电动出租车达到最佳的运营表现，政府在推广电动出租车的过程中应当建设更多高速率的充电站以提升电动出租车的运营表现。

关键词: 深度强化学习, 电动出租车, DDQN, 出租车运营

Abstract: Electric taxis have been demonstrated with the promotion of electric vehicles. Compared with internal combustion engine vehicles, electric taxis spend more time in recharging, which reduces the taxi drivers’ intention to use. Reinforcement learning is applicable to the sequential decision-making process of taxis driver. This paper presents the double deep Q-learning network (DDQN) model to simulate the operation of electric taxis. According to the real-time state of taxis, DDQN will choose the optimal actions to execute. After training, we obtain a global optimal electric taxi service strategy, and finally optimize the taxi service. Using real-world taxi travel data, an experiment is conducted in Manhattan Island in New York City, USA. Results show that, comparing with the baseline methods, DDQN reduces the waiting time for charging and the rejection rate by 70% and 53%, respectively. Taxi drives’ income are finally increased by about 7%. Moreover, the results of model parameter sensitivity analysis indicate that the charge speed and the number of vehicles have greater impact on drives’ income than the battery capacity. When the charging rate reaches 120 kW, electric taxis achieve the best performance. The government should build more fast charging station to improve the revenue of electric taxis.

Key words: deep reinforcement learning, electric taxi, DDQN, taxi service strategies

中图分类号:

P208

叶浩宇, 涂伟, 叶贺辉, 麦可, 赵天鸿, 李清泉. 基于深度强化学习的电动出租车运营优化[J]. 测绘学报, 2020, 49(12): 1630-1639.

YE Haoyu, TU Wei, YE Hehui, MAI Ke, ZHAO Tianhong, LI Qingquan. Deep reinforcement learning based electric taxi service optimization[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(12): 1630-1639.

参考文献

[1] 施晓清, 李笑诺, 杨建新. 低碳交通电动汽车碳减排潜力及其影响因素分析[J]. 环境科学, 2013, 34(1):385-394. SHI Xiaoqing, LI Xiaonuo, YANG Jianxin. Research on carbon reduction potential of electric vehicles for low-carbon transportation and its influencing factors[J]. Chinese Journal of Environmental Science, 2013, 34(1):385-394.
[2] 高云. 巴黎气候变化大会后中国的气候变化应对形势[J]. 气候变化研究进展, 2017, 13(1):89-94. GAO Yun. China's response to climate change issues after Paris_Climate Change Conference[J]. Climate Change Research, 2017, 13(1):89-94.
[3] 李德仁. 展望大数据时代的地球空间信息学[J]. 测绘学报, 2016, 45(4):379-384. DOI:10.11947/j.AGCS.2016.20160057. LI Deren. Towards geo-spatial information science in big data era[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(4):379-384. DOI:10.11947/j.AGCS.2016.20160057.
[4] 吴华意, 黄蕊, 游兰, 等. 出租车轨迹数据挖掘进展[J]. 测绘学报, 2019, 48(11):1341-1356. DOI:10.11947/j.AGCS.2019.20190210. WU Huayi, HUANG Rui, YOU Lan, et al. Recent progress in taxi trajectory data mining[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(11):1341-1356. DOI:10.11947/j.AGCS.2019.20190210.
[5] 中国公路学报编辑部. 中国汽车工程学术研究综述·2017[J]. 中国公路学报, 2017, 30(6):1-197. Editorial Department of China Journal of Highway and Transport. Review on China's automotive engineering research progress:2017[J]. China Journal of Highway and Transport, 2017, 30(6):1-197.
[6] 李清泉. 从Geomatics到Urban Informatics[J]. 武汉大学学报(信息科学版), 2017, 42(1):1-6. LI Qingquan. From geomatics to urban informatics[J]. Geomatics and Information Science of Wuhan University, 2017, 42(1):1-6.
[7] 李德仁, 李清泉, 杨必胜, 等. 3S技术与智能交通[J]. 武汉大学学报(信息科学版), 2008, 33(4):331-336. LI Deren, LI Qingquan, YANG Bisheng, et al. Techniques of GIS, GPS and RS for the development of intelligent transportation[J]. Geomatics and Information Science of Wuhan University, 2008, 33(4):331-336.
[8] 涂伟, 李清泉, 方志祥. 一种大规模车辆路径问题的启发式算法[J]. 武汉大学学报(信息科学版),2013, 38(3):307-310, 338. TU Wei, LI Qingquan, FANG Zhixiang. A heuristic algorithm for large scale vehicle routing problem[J]. Geomatics and Information Science of Wuhan University, 2013, 38(3):307-310, 338.
[9] 唐炉亮, 阚子涵, 任畅, 等. 利用GPS轨迹的转向级交通拥堵精细分析[J]. 测绘学报, 2019, 48(1):75-85. DOI:10.11947/j.AGCS.2019.20170448. TANG Luliang, KAN Zihan, REN Chang, et al. Fine-grained analysis of traffic congestions at the turning level using GPS traces[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1):75-85. DOI:10.11947/j.AGCS.2019.20170448.
[10] VAZIFEH M M, SANTI P, RESTA G, et al. Addressing the minimum fleet problem in on-demand urban mobility[J]. Nature, 2018, 557(7706):534-538.
[11] ICCT. Electric vehicle capitals:Accelerating the global transition to electric drive[C]//Proceedings of International Council on Clean Transportation.[S.l.]:ICCT, 2018.
[12] TU Wei, LI Qingquan, FANG Zhixiang, et al. Optimizing the locations of electric taxi charging stations:a spatial-temporal demand coverage approach[J]. Transportation Research Part C:Emerging Technologies, 2016, 65:172-189.
[13] GAN Lingwen, TOPCU U, LOW S H. Optimal decentralized protocol for electric vehicle charging[J]. IEEE Transactions on Power Systems, 2012, 28(2):940-951.
[14] KIM H J, LEE J, PARK G L, et al. An efficient scheduling scheme on charging stations for smart transportation[C]//Proceedings of International Conference on Security-Enriched Urban Computing and Smart Grid.Berlin:Springer, 2010, 78:274-278.
[15] TIAN Zhiyong, JUNG T, WANG Yi, et al. Real-time charging station recommendation system for electric-vehicle taxis[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(11):3098-3109.
[16] TU Wei, MAI Ke, ZHANG Yatao, et al. Real-time route recommendations for E-taxies leveraging GPS trajectories[J]. IEEE Transactions on Industrial Informatics,2020. DOI:10.1109/TⅡ.2020.2990206.
[17] KAELBLING L P, LITTMAN M L, MOORE A P. Reinforcement learning:a survey[J]. Journal of Artificial Intelligence Research, 1996, 4(1):237-285.
[18] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540):529-533.
[19] KOBER J, BAGNELL J A, PETERS J. Reinforcement learning in robotics:a survey[J]. The International Journal of Robotics Research, 2013, 32(11):1238-1274.
[20] GAO Yong,JIANG Dan, XU Yan. Optimize taxi driving strategies based on reinforcement learning[J]. International Journal of Geographical Information Science, 2018, 32(8):1677-1696.
[21] VERMA T, VARAKANTHAM P, KRAUS S, et al. Augmenting decisions of taxi drivers through reinforcement learning for improving revenues[C]//Proceedings of the Twenty-Seventh International Conference on Automated Planning and Scheduling (ICAPS 2017).[S.l.]:ICAPS, 2017:409-417.
[22] 荆朝霞, 郭文骏, 郭子暄. 基于多代理技术的电动出租车运营实时仿真系统及应用[J]. 电力系统自动化, 2016, 40(7):83-89. JING Zhaoxia, GUO Wenjun, GUO Zixuan. Real-time simulation of electric taxi operation based on multi-agent technology[J]. Automation of Electric Power Systems, 2016, 40(7):83-89.
[23] TSENG C M, CHAU S C K, LIU Xue. Improving viability of electric taxis by taxi service strategy optimization:a big data study of New York city[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 20(3):817-829.
[24] 高阳, 陈世福, 陆鑫. 强化学习研究综述[J]. 自动化学报, 2004, 30(1):86-100. GAO Yang, CHEN Shifu, LU Xin. Research on reinforcement learning technology:a review[J]. Acta Automatica Sinica, 2004, 30(1):86-100.
[25] BELLMAN R. Dynamic programming and lagrange multipliers[J]. Proceedings of the National Academy of Sciences of the United States of America, 1956, 42(10):767-769.
[26] WATKINS C J C H, DAYAN P. Technical note:Q-learning[J]. Machine Learning, 1992, 8(3-4):279-292.
[27] TESAURO G. Temporal difference learning and TD-Gammon[J]. Communications of the ACM, 1995, 38(3):58-68.
[28] VAN HASSELT H, GUEZ A, SILVER D. Deep reinforcement learning with double Q-learning[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence.[S. l.]:AAAI Press, 2016:302-315.
[29] LIN L J. Reinforcement learning for robots using neural networks[D]. Schenley Park Pittsburgh, PA, United States:Carnegie Mellon University, 1993.
[30] BAUER G S, GREENBLATT J B, GERKE B F. Cost, energy, and environmental impact of automated electric taxi fleets in Manhattan[J]. Environmental Science & Technology, 2018, 52(8):4920-4928.

基于深度强化学习的电动出租车运营优化

Deep reinforcement learning based electric taxi service optimization

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	刘经南, 罗亚荣, 郭迟, 高柯夫. PNT智能与智能PNT[J]. 测绘学报, 2022, 51(6): 811-828.
[2]	韩李涛, 周丽娟, 龚城, 张爱国. 顾及步行习惯的室内导航网络及其生成算法[J]. 测绘学报, 2022, 51(5): 729-738.
[3]	方金凤, 孟祥福. 基于LBSN和多图融合的兴趣点推荐[J]. 测绘学报, 2022, 51(5): 739-749.
[4]	张睿卓. 基于多源数据的林区电力走廊安全风险评估方法[J]. 测绘学报, 2022, 51(5): 784-784.
[5]	夏吉喆, 周颖, 李珍, 李帆, 乐阳, 程涛, 李清泉. 城市时空大数据驱动的新型冠状病毒传播风险评估——以粤港澳大湾区为例[J]. 测绘学报, 2020, 49(6): 671-680.
[6]	朱庆, 冯斌, 李茂粟, 陈媚特, 徐肇文, 谢潇, 张叶廷, 刘铭崴, 黄志勤, 冯义从. 面向动态关联数据的高效稀疏图索引方法[J]. 测绘学报, 2020, 49(6): 681-691.
[7]	陆川伟, 孙群, 陈冰, 温伯威, 赵云鹏, 徐立. 车辆轨迹数据的道路学习提取法[J]. 测绘学报, 2020, 49(6): 692-702.
[8]	尹烁, 闫小明, 晏雄锋. 基于特征边重构的建筑物化简方法[J]. 测绘学报, 2020, 49(6): 703-710.
[9]	吴华意, 黄蕊, 游兰, 向隆刚. 出租车轨迹数据挖掘进展[J]. 测绘学报, 2019, 48(11): 1341-1356.
[10]	郭庆胜, 刘洋, 李萌, 程晓茜, 何捷, 王慧慧, 魏智威. 基于网格模型的导航道路图渐进式化简方法[J]. 测绘学报, 2019, 48(11): 1357-1368.
[11]	吴政, 武鹏达, 李成名. 对等网络下自适应层级的矢量数据时空索引构建方法[J]. 测绘学报, 2019, 48(11): 1369-1379.
[12]	王培晓, 张恒才, 王海波, 吴升. ST-CFSFDP:快速搜索密度峰值的时空聚类算法[J]. 测绘学报, 2019, 48(11): 1380-1390.
[13]	万子健, 李连营, 杨敏, 周校东. 车辆轨迹数据提取道路交叉口特征的决策树模型[J]. 测绘学报, 2019, 48(11): 1391-1403.
[14]	胡光辉, 熊礼阳, 汤国安. DEM地表坡向变率的向量几何计算法[J]. 测绘学报, 2019, 48(11): 1404-1414.
[15]	张旭, 郝向阳, 李建胜, 李朋月. 监控视频中动态目标与地理空间信息的融合与可视化方法[J]. 测绘学报, 2019, 48(11): 1415-1423.