基于深度强化学习的电动出租车运营优化

doi:10.11947/j.AGCS.2020.20190516

测绘学报 ›› 2020, Vol. 49 ›› Issue (12): 1630-1639.doi: 10.11947/j.AGCS.2020.20190516

基于深度强化学习的电动出租车运营优化

叶浩宇¹, 涂伟^2,3,4,5, 叶贺辉⁶, 麦可^2,3,4, 赵天鸿^2,4, 李清泉^1,2,3,4

1. 武汉大学测绘遥感信息工程国家重点实验室, 湖北武汉 430079;
2. 深圳大学建筑与城市规划学院城市空间信息工程系, 广东深圳 518060;
3. 人工智能与数字经济广东省实验室(深圳), 广东深圳 518060;
4. 广东省城市空间信息工程重点实验室, 广东深圳 518060;
5. 自然资源部大湾区地理环境监测重点实验室, 广东深圳 518060;
6. 闽江学院软件学院, 福建福州 350108

收稿日期:2019-12-16 修回日期:2020-06-07 发布日期:2020-12-25
通讯作者: 涂伟 E-mail:tuwei@szu.edu.cn
作者简介:叶浩宇(1996-),男,硕士生,研究方向为城市时空大数据分析。E-mail:yehaoyuchn@whu.edu.cn
基金资助:
国家重点研发计划（2019YFB2103104）；广东省自然科学基金（2019A1515011049）；深圳市科技创新委员会基础研究（JCYJ20170412105839839）

Deep reinforcement learning based electric taxi service optimization

YE Haoyu¹, TU Wei^2,3,4,5, YE Hehui⁶, MAI Ke^2,3,4, ZHAO Tianhong^2,4, LI Qingquan^1,2,3,4

1. State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan 430079, China;
2. Department of Urban Informatics, School of Architecture and Urban Planning, Shenzhen University, Shenzhen 518060, China;
3. Guangdong Laboratory of Artificial Intelligence and Digital Economy(SZ), Shenzhen University, Shenzhen 518060, China;
4. Guangdong Key Laboratory of Urban Informatics, Shenzhen University, Shenzhen 518060, China;
5. Key Laboratory for Geo-Environmental Monitoring of Great Bay Area, MNR, Shenzhen 518060, China;
6. Software College, Minjiang University, Fuzhou 350108, China

Received:2019-12-16 Revised:2020-06-07 Published:2020-12-25
Supported by:
The National Key Research and Development Program of China (No. 2019YFB2103104);The Natural Science Foundation of Guangdong Province(No. 2019A1515011049);The Basic Research Projects of Shenzhen Technology Innovation Commission (No. JCYJ20170412105839839)

摘要/Abstract

摘要： 作为公共交通的重要组成部分，电动出租车对电动车推广具有重要的示范意义。相较于燃油出租车，电动出租车需要耗费更多充电时间，降低了出租车司机的使用意愿，全面推广面临较大阻力。强化学习方法方兴未艾，适用于出租车运营的顺序决策过程。基于强化学习，本文构建双深度Q学习网络（double deep Q-learning network，DDQN）模型模拟电动出租车的运行。根据出租车的实时状态选择并执行最优载客、充电、空驶和等待等动作，通过训练得到全局最优的电动出租车运营策略，实现电动出租车运营智能优化。利用美国纽约市曼哈顿岛的出租车出行数据进行试验。结果表明：相较于简单的电动出租车运营模式，DDQN优化策略最高将充电等待时长降低70%，拒载率降低53%，司机的载客收入提高7%。相对于电池容量，充电速率和车辆总数对出租车运营收入影响更大，当充电速率达到120 kW时，电动出租车达到最佳的运营表现，政府在推广电动出租车的过程中应当建设更多高速率的充电站以提升电动出租车的运营表现。

关键词: 深度强化学习, 电动出租车, DDQN, 出租车运营

Abstract: Electric taxis have been demonstrated with the promotion of electric vehicles. Compared with internal combustion engine vehicles, electric taxis spend more time in recharging, which reduces the taxi drivers’ intention to use. Reinforcement learning is applicable to the sequential decision-making process of taxis driver. This paper presents the double deep Q-learning network (DDQN) model to simulate the operation of electric taxis. According to the real-time state of taxis, DDQN will choose the optimal actions to execute. After training, we obtain a global optimal electric taxi service strategy, and finally optimize the taxi service. Using real-world taxi travel data, an experiment is conducted in Manhattan Island in New York City, USA. Results show that, comparing with the baseline methods, DDQN reduces the waiting time for charging and the rejection rate by 70% and 53%, respectively. Taxi drives’ income are finally increased by about 7%. Moreover, the results of model parameter sensitivity analysis indicate that the charge speed and the number of vehicles have greater impact on drives’ income than the battery capacity. When the charging rate reaches 120 kW, electric taxis achieve the best performance. The government should build more fast charging station to improve the revenue of electric taxis.

Key words: deep reinforcement learning, electric taxi, DDQN, taxi service strategies

中图分类号:

P208

叶浩宇, 涂伟, 叶贺辉, 麦可, 赵天鸿, 李清泉. 基于深度强化学习的电动出租车运营优化[J]. 测绘学报, 2020, 49(12): 1630-1639.

YE Haoyu, TU Wei, YE Hehui, MAI Ke, ZHAO Tianhong, LI Qingquan. Deep reinforcement learning based electric taxi service optimization[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(12): 1630-1639.

参考文献

[1] 施晓清, 李笑诺, 杨建新. 低碳交通电动汽车碳减排潜力及其影响因素分析[J]. 环境科学, 2013, 34(1):385-394. SHI Xiaoqing, LI Xiaonuo, YANG Jianxin. Research on carbon reduction potential of electric vehicles for low-carbon transportation and its influencing factors[J]. Chinese Journal of Environmental Science, 2013, 34(1):385-394.
[2] 高云. 巴黎气候变化大会后中国的气候变化应对形势[J]. 气候变化研究进展, 2017, 13(1):89-94. GAO Yun. China's response to climate change issues after Paris_Climate Change Conference[J]. Climate Change Research, 2017, 13(1):89-94.
[3] 李德仁. 展望大数据时代的地球空间信息学[J]. 测绘学报, 2016, 45(4):379-384. DOI:10.11947/j.AGCS.2016.20160057. LI Deren. Towards geo-spatial information science in big data era[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(4):379-384. DOI:10.11947/j.AGCS.2016.20160057.
[4] 吴华意, 黄蕊, 游兰, 等. 出租车轨迹数据挖掘进展[J]. 测绘学报, 2019, 48(11):1341-1356. DOI:10.11947/j.AGCS.2019.20190210. WU Huayi, HUANG Rui, YOU Lan, et al. Recent progress in taxi trajectory data mining[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(11):1341-1356. DOI:10.11947/j.AGCS.2019.20190210.
[5] 中国公路学报编辑部. 中国汽车工程学术研究综述·2017[J]. 中国公路学报, 2017, 30(6):1-197. Editorial Department of China Journal of Highway and Transport. Review on China's automotive engineering research progress:2017[J]. China Journal of Highway and Transport, 2017, 30(6):1-197.
[6] 李清泉. 从Geomatics到Urban Informatics[J]. 武汉大学学报(信息科学版), 2017, 42(1):1-6. LI Qingquan. From geomatics to urban informatics[J]. Geomatics and Information Science of Wuhan University, 2017, 42(1):1-6.
[7] 李德仁, 李清泉, 杨必胜, 等. 3S技术与智能交通[J]. 武汉大学学报(信息科学版), 2008, 33(4):331-336. LI Deren, LI Qingquan, YANG Bisheng, et al. Techniques of GIS, GPS and RS for the development of intelligent transportation[J]. Geomatics and Information Science of Wuhan University, 2008, 33(4):331-336.
[8] 涂伟, 李清泉, 方志祥. 一种大规模车辆路径问题的启发式算法[J]. 武汉大学学报(信息科学版),2013, 38(3):307-310, 338. TU Wei, LI Qingquan, FANG Zhixiang. A heuristic algorithm for large scale vehicle routing problem[J]. Geomatics and Information Science of Wuhan University, 2013, 38(3):307-310, 338.
[9] 唐炉亮, 阚子涵, 任畅, 等. 利用GPS轨迹的转向级交通拥堵精细分析[J]. 测绘学报, 2019, 48(1):75-85. DOI:10.11947/j.AGCS.2019.20170448. TANG Luliang, KAN Zihan, REN Chang, et al. Fine-grained analysis of traffic congestions at the turning level using GPS traces[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1):75-85. DOI:10.11947/j.AGCS.2019.20170448.
[10] VAZIFEH M M, SANTI P, RESTA G, et al. Addressing the minimum fleet problem in on-demand urban mobility[J]. Nature, 2018, 557(7706):534-538.
[11] ICCT. Electric vehicle capitals:Accelerating the global transition to electric drive[C]//Proceedings of International Council on Clean Transportation.[S.l.]:ICCT, 2018.
[12] TU Wei, LI Qingquan, FANG Zhixiang, et al. Optimizing the locations of electric taxi charging stations:a spatial-temporal demand coverage approach[J]. Transportation Research Part C:Emerging Technologies, 2016, 65:172-189.
[13] GAN Lingwen, TOPCU U, LOW S H. Optimal decentralized protocol for electric vehicle charging[J]. IEEE Transactions on Power Systems, 2012, 28(2):940-951.
[14] KIM H J, LEE J, PARK G L, et al. An efficient scheduling scheme on charging stations for smart transportation[C]//Proceedings of International Conference on Security-Enriched Urban Computing and Smart Grid.Berlin:Springer, 2010, 78:274-278.
[15] TIAN Zhiyong, JUNG T, WANG Yi, et al. Real-time charging station recommendation system for electric-vehicle taxis[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(11):3098-3109.
[16] TU Wei, MAI Ke, ZHANG Yatao, et al. Real-time route recommendations for E-taxies leveraging GPS trajectories[J]. IEEE Transactions on Industrial Informatics,2020. DOI:10.1109/TⅡ.2020.2990206.
[17] KAELBLING L P, LITTMAN M L, MOORE A P. Reinforcement learning:a survey[J]. Journal of Artificial Intelligence Research, 1996, 4(1):237-285.
[18] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540):529-533.
[19] KOBER J, BAGNELL J A, PETERS J. Reinforcement learning in robotics:a survey[J]. The International Journal of Robotics Research, 2013, 32(11):1238-1274.
[20] GAO Yong,JIANG Dan, XU Yan. Optimize taxi driving strategies based on reinforcement learning[J]. International Journal of Geographical Information Science, 2018, 32(8):1677-1696.
[21] VERMA T, VARAKANTHAM P, KRAUS S, et al. Augmenting decisions of taxi drivers through reinforcement learning for improving revenues[C]//Proceedings of the Twenty-Seventh International Conference on Automated Planning and Scheduling (ICAPS 2017).[S.l.]:ICAPS, 2017:409-417.
[22] 荆朝霞, 郭文骏, 郭子暄. 基于多代理技术的电动出租车运营实时仿真系统及应用[J]. 电力系统自动化, 2016, 40(7):83-89. JING Zhaoxia, GUO Wenjun, GUO Zixuan. Real-time simulation of electric taxi operation based on multi-agent technology[J]. Automation of Electric Power Systems, 2016, 40(7):83-89.
[23] TSENG C M, CHAU S C K, LIU Xue. Improving viability of electric taxis by taxi service strategy optimization:a big data study of New York city[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 20(3):817-829.
[24] 高阳, 陈世福, 陆鑫. 强化学习研究综述[J]. 自动化学报, 2004, 30(1):86-100. GAO Yang, CHEN Shifu, LU Xin. Research on reinforcement learning technology:a review[J]. Acta Automatica Sinica, 2004, 30(1):86-100.
[25] BELLMAN R. Dynamic programming and lagrange multipliers[J]. Proceedings of the National Academy of Sciences of the United States of America, 1956, 42(10):767-769.
[26] WATKINS C J C H, DAYAN P. Technical note:Q-learning[J]. Machine Learning, 1992, 8(3-4):279-292.
[27] TESAURO G. Temporal difference learning and TD-Gammon[J]. Communications of the ACM, 1995, 38(3):58-68.
[28] VAN HASSELT H, GUEZ A, SILVER D. Deep reinforcement learning with double Q-learning[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence.[S. l.]:AAAI Press, 2016:302-315.
[29] LIN L J. Reinforcement learning for robots using neural networks[D]. Schenley Park Pittsburgh, PA, United States:Carnegie Mellon University, 1993.
[30] BAUER G S, GREENBLATT J B, GERKE B F. Cost, energy, and environmental impact of automated electric taxi fleets in Manhattan[J]. Environmental Science & Technology, 2018, 52(8):4920-4928.

基于深度强化学习的电动出租车运营优化

Deep reinforcement learning based electric taxi service optimization

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	邱越, 武芳, 翟仁健, 钱海忠, 黄哲琨, 李博. 面向匹配优化的多源建筑物实体级保形空间对齐模型[J]. 测绘学报, 2025, 54(12): 2262-2275.
[2]	张锦彬, 朱军, 党沛, 周宇轩, 杨博文. 现场直播式地理信息服务：基于VR全景的现场实况远程临浸感知[J]. 测绘学报, 2025, 54(12): 2276-2286.
[3]	张岩. 基于街景影像的城市功能区多尺度时空感知方法[J]. 测绘学报, 2025, 54(12): 2289-2289.
[4]	曾进. 城市社会空间的空间大数据量化表达与分析方法：以深圳市为例[J]. 测绘学报, 2025, 54(12): 2292-2292.
[5]	刘少俊. 基于手机信令数据的城市人群活动时空格局分析研究[J]. 测绘学报, 2025, 54(12): 2295-2295.
[6]	吴超, 梁咏翔, 岳瀚, 崔远政, 黄波. 面向计数数据的时空地理加权泊松回归模型[J]. 测绘学报, 2025, 54(11): 2026-2039.
[7]	王小龙, 王卓, 李精忠, 闫浩文. 微地图制图的空间方向关系转译法[J]. 测绘学报, 2025, 54(11): 2040-2051.
[8]	胡鑫, 杨学习, 江一凡, 王宪彬, 丁晨, 谢顾然, 邓敏. 基于多智能体层次化协同的地理事件抽取与时空解析[J]. 测绘学报, 2025, 54(11): 2052-2067.
[9]	李俊, 李朝奎, 黄磊, 冯媛媛. 高速公路广告牌巡检目标跟踪的改进ByteTrack算法[J]. 测绘学报, 2025, 54(11): 2068-2080.
[10]	叶欣宇, 徐胜华, 刘纪平, 陈虹宇, 王琢璐, 李维炼. 基于时空因果推断的下一个兴趣点推荐[J]. 测绘学报, 2025, 54(11): 2081-2096.
[11]	赵学胜, 谢文澜, 孙文彬. 空间格网互操作的研究进展与关键问题[J]. 测绘学报, 2025, 54(10): 1727-1740.
[12]	高凡, 路威, 甘麟露, 章繁, 荣凤娟, 汤士涵. 智能驱动的并行地理计算引擎框架[J]. 测绘学报, 2025, 54(10): 1877-1892.
[13]	吴浩宇, 朱庆, 丁雨淋, 鲍榴, 刘利. 数据模型知识协同驱动的隧道围岩高精度数字孪生建模方法[J]. 测绘学报, 2025, 54(10): 1893-1906.
[14]	郝彧露. 时空数据驱动的城市区域火灾风险评估预测模型及应用[J]. 测绘学报, 2025, 54(10): 1910-1910.
[15]	张付兵, 孙群, 徐青, 马京振, 黄文君, 陈若虚. 随机森林和图神经网络支持下的河系自动分级与选取方法[J]. 测绘学报, 2025, 54(9): 1697-1711.