Deep reinforcement learning based electric taxi service optimization

doi:10.11947/j.AGCS.2020.20190516

Abstract

Abstract: Electric taxis have been demonstrated with the promotion of electric vehicles. Compared with internal combustion engine vehicles, electric taxis spend more time in recharging, which reduces the taxi drivers’ intention to use. Reinforcement learning is applicable to the sequential decision-making process of taxis driver. This paper presents the double deep Q-learning network (DDQN) model to simulate the operation of electric taxis. According to the real-time state of taxis, DDQN will choose the optimal actions to execute. After training, we obtain a global optimal electric taxi service strategy, and finally optimize the taxi service. Using real-world taxi travel data, an experiment is conducted in Manhattan Island in New York City, USA. Results show that, comparing with the baseline methods, DDQN reduces the waiting time for charging and the rejection rate by 70% and 53%, respectively. Taxi drives’ income are finally increased by about 7%. Moreover, the results of model parameter sensitivity analysis indicate that the charge speed and the number of vehicles have greater impact on drives’ income than the battery capacity. When the charging rate reaches 120 kW, electric taxis achieve the best performance. The government should build more fast charging station to improve the revenue of electric taxis.

Key words: deep reinforcement learning, electric taxi, DDQN, taxi service strategies

CLC Number:

P208

YE Haoyu, TU Wei, YE Hehui, MAI Ke, ZHAO Tianhong, LI Qingquan. Deep reinforcement learning based electric taxi service optimization[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(12): 1630-1639.

References

[1] 施晓清, 李笑诺, 杨建新. 低碳交通电动汽车碳减排潜力及其影响因素分析[J]. 环境科学, 2013, 34(1):385-394. SHI Xiaoqing, LI Xiaonuo, YANG Jianxin. Research on carbon reduction potential of electric vehicles for low-carbon transportation and its influencing factors[J]. Chinese Journal of Environmental Science, 2013, 34(1):385-394.
[2] 高云. 巴黎气候变化大会后中国的气候变化应对形势[J]. 气候变化研究进展, 2017, 13(1):89-94. GAO Yun. China's response to climate change issues after Paris_Climate Change Conference[J]. Climate Change Research, 2017, 13(1):89-94.
[3] 李德仁. 展望大数据时代的地球空间信息学[J]. 测绘学报, 2016, 45(4):379-384. DOI:10.11947/j.AGCS.2016.20160057. LI Deren. Towards geo-spatial information science in big data era[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(4):379-384. DOI:10.11947/j.AGCS.2016.20160057.
[4] 吴华意, 黄蕊, 游兰, 等. 出租车轨迹数据挖掘进展[J]. 测绘学报, 2019, 48(11):1341-1356. DOI:10.11947/j.AGCS.2019.20190210. WU Huayi, HUANG Rui, YOU Lan, et al. Recent progress in taxi trajectory data mining[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(11):1341-1356. DOI:10.11947/j.AGCS.2019.20190210.
[5] 中国公路学报编辑部. 中国汽车工程学术研究综述·2017[J]. 中国公路学报, 2017, 30(6):1-197. Editorial Department of China Journal of Highway and Transport. Review on China's automotive engineering research progress:2017[J]. China Journal of Highway and Transport, 2017, 30(6):1-197.
[6] 李清泉. 从Geomatics到Urban Informatics[J]. 武汉大学学报(信息科学版), 2017, 42(1):1-6. LI Qingquan. From geomatics to urban informatics[J]. Geomatics and Information Science of Wuhan University, 2017, 42(1):1-6.
[7] 李德仁, 李清泉, 杨必胜, 等. 3S技术与智能交通[J]. 武汉大学学报(信息科学版), 2008, 33(4):331-336. LI Deren, LI Qingquan, YANG Bisheng, et al. Techniques of GIS, GPS and RS for the development of intelligent transportation[J]. Geomatics and Information Science of Wuhan University, 2008, 33(4):331-336.
[8] 涂伟, 李清泉, 方志祥. 一种大规模车辆路径问题的启发式算法[J]. 武汉大学学报(信息科学版),2013, 38(3):307-310, 338. TU Wei, LI Qingquan, FANG Zhixiang. A heuristic algorithm for large scale vehicle routing problem[J]. Geomatics and Information Science of Wuhan University, 2013, 38(3):307-310, 338.
[9] 唐炉亮, 阚子涵, 任畅, 等. 利用GPS轨迹的转向级交通拥堵精细分析[J]. 测绘学报, 2019, 48(1):75-85. DOI:10.11947/j.AGCS.2019.20170448. TANG Luliang, KAN Zihan, REN Chang, et al. Fine-grained analysis of traffic congestions at the turning level using GPS traces[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1):75-85. DOI:10.11947/j.AGCS.2019.20170448.
[10] VAZIFEH M M, SANTI P, RESTA G, et al. Addressing the minimum fleet problem in on-demand urban mobility[J]. Nature, 2018, 557(7706):534-538.
[11] ICCT. Electric vehicle capitals:Accelerating the global transition to electric drive[C]//Proceedings of International Council on Clean Transportation.[S.l.]:ICCT, 2018.
[12] TU Wei, LI Qingquan, FANG Zhixiang, et al. Optimizing the locations of electric taxi charging stations:a spatial-temporal demand coverage approach[J]. Transportation Research Part C:Emerging Technologies, 2016, 65:172-189.
[13] GAN Lingwen, TOPCU U, LOW S H. Optimal decentralized protocol for electric vehicle charging[J]. IEEE Transactions on Power Systems, 2012, 28(2):940-951.
[14] KIM H J, LEE J, PARK G L, et al. An efficient scheduling scheme on charging stations for smart transportation[C]//Proceedings of International Conference on Security-Enriched Urban Computing and Smart Grid.Berlin:Springer, 2010, 78:274-278.
[15] TIAN Zhiyong, JUNG T, WANG Yi, et al. Real-time charging station recommendation system for electric-vehicle taxis[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(11):3098-3109.
[16] TU Wei, MAI Ke, ZHANG Yatao, et al. Real-time route recommendations for E-taxies leveraging GPS trajectories[J]. IEEE Transactions on Industrial Informatics,2020. DOI:10.1109/TⅡ.2020.2990206.
[17] KAELBLING L P, LITTMAN M L, MOORE A P. Reinforcement learning:a survey[J]. Journal of Artificial Intelligence Research, 1996, 4(1):237-285.
[18] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540):529-533.
[19] KOBER J, BAGNELL J A, PETERS J. Reinforcement learning in robotics:a survey[J]. The International Journal of Robotics Research, 2013, 32(11):1238-1274.
[20] GAO Yong,JIANG Dan, XU Yan. Optimize taxi driving strategies based on reinforcement learning[J]. International Journal of Geographical Information Science, 2018, 32(8):1677-1696.
[21] VERMA T, VARAKANTHAM P, KRAUS S, et al. Augmenting decisions of taxi drivers through reinforcement learning for improving revenues[C]//Proceedings of the Twenty-Seventh International Conference on Automated Planning and Scheduling (ICAPS 2017).[S.l.]:ICAPS, 2017:409-417.
[22] 荆朝霞, 郭文骏, 郭子暄. 基于多代理技术的电动出租车运营实时仿真系统及应用[J]. 电力系统自动化, 2016, 40(7):83-89. JING Zhaoxia, GUO Wenjun, GUO Zixuan. Real-time simulation of electric taxi operation based on multi-agent technology[J]. Automation of Electric Power Systems, 2016, 40(7):83-89.
[23] TSENG C M, CHAU S C K, LIU Xue. Improving viability of electric taxis by taxi service strategy optimization:a big data study of New York city[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 20(3):817-829.
[24] 高阳, 陈世福, 陆鑫. 强化学习研究综述[J]. 自动化学报, 2004, 30(1):86-100. GAO Yang, CHEN Shifu, LU Xin. Research on reinforcement learning technology:a review[J]. Acta Automatica Sinica, 2004, 30(1):86-100.
[25] BELLMAN R. Dynamic programming and lagrange multipliers[J]. Proceedings of the National Academy of Sciences of the United States of America, 1956, 42(10):767-769.
[26] WATKINS C J C H, DAYAN P. Technical note:Q-learning[J]. Machine Learning, 1992, 8(3-4):279-292.
[27] TESAURO G. Temporal difference learning and TD-Gammon[J]. Communications of the ACM, 1995, 38(3):58-68.
[28] VAN HASSELT H, GUEZ A, SILVER D. Deep reinforcement learning with double Q-learning[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence.[S. l.]:AAAI Press, 2016:302-315.
[29] LIN L J. Reinforcement learning for robots using neural networks[D]. Schenley Park Pittsburgh, PA, United States:Carnegie Mellon University, 1993.
[30] BAUER G S, GREENBLATT J B, GERKE B F. Cost, energy, and environmental impact of automated electric taxi fleets in Manhattan[J]. Environmental Science & Technology, 2018, 52(8):4920-4928.