HMDRL: Hierarchical Mixed Deep Reinforcement Learning to Balance Vehicle Supply and Demand
Xi, Jinhao1,2; Zhu, Fenghua1,2; Ye, Peijun1,2; Lv, Yisheng1,2; Tang, Haina2; Wang, Fei-Yue1,2
Source PublicationIEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
ISSN1524-9050
2022-07-25
Pages12
Corresponding AuthorZhu, Fenghua(fenghua.zhu@ia.ac.cn) ; Ye, Peijun(peijun_ye@hotmail.com)
AbstractThe imbalance of vehicle supply and demand is a common phenomenon that influences the efficiency of online ride-hailing systems greatly. To address this problem, a three-level hierarchical mixed deep reinforcement learning method (HMDRL) is proposed to reposition idle vehicles. A manager operates at the top level, where action-abstraction is conducted from the time dimension and is adaptive for spatially scalable and time-varying systems. Coordinators locate at the middle level and a parallel coordination mechanism that is independent of the decision order is designed to improve the efficiency of the repositioning. The bottom level is composed of executive workers to reposition vehicles with mixed states and the states contain spatiotemporal information of agents' neighbor areas. Two reward functions are designed for the manager and the coordinators, respectively, aiming to improve the training effect by avoiding sparse rewards. A simulator based on real orders is designed and HMDRL is compared with six methods. Experimental results demonstrate that HMDRL outperforms all the other methods. In three comparison experiments, the order response rate is increased by 0.62% to 8.29%, 1.5% to 7.78%, 1.18% to 4.75%, respectively.
KeywordReinforcement learning Pricing Heuristic algorithms Supply and demand Surges Vehicle dynamics Vehicles Deep reinforcement learning online ride-hailing system hierarchical repositioning framework parallel coordination mechanism mixed state
DOI10.1109/TITS.2022.3191752
Indexed BySCI
Language英语
Funding ProjectKey-Area Research and Development Program of Guangdong Province[2020B0909050001] ; National Natural Science Foundation of China (NSFC)[U1811463] ; National Natural Science Foundation of China (NSFC)[U1909204] ; National Natural Science Foundation of China (NSFC)[61876011] ; National Natural Science Foundation of China (NSFC)[52071312] ; Chinese Academy of Sciences (CAS) Science and Technology Service Network Plan (STS) Dongguan Project[20201600200132]
Funding OrganizationKey-Area Research and Development Program of Guangdong Province ; National Natural Science Foundation of China (NSFC) ; Chinese Academy of Sciences (CAS) Science and Technology Service Network Plan (STS) Dongguan Project
WOS Research AreaEngineering ; Transportation
WOS SubjectEngineering, Civil ; Engineering, Electrical & Electronic ; Transportation Science & Technology
WOS IDWOS:000833052300001
PublisherIEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Citation statistics
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/49762
Collection复杂系统管理与控制国家重点实验室_平行智能技术与系统团队
Corresponding AuthorZhu, Fenghua; Ye, Peijun
Affiliation1.Chinese Acad Sci, State Key Lab Management & Control Complex Syst, Inst Automat, Beijing 100190, Peoples R China
2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China
First Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Corresponding Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Xi, Jinhao,Zhu, Fenghua,Ye, Peijun,et al. HMDRL: Hierarchical Mixed Deep Reinforcement Learning to Balance Vehicle Supply and Demand[J]. IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,2022:12.
APA Xi, Jinhao,Zhu, Fenghua,Ye, Peijun,Lv, Yisheng,Tang, Haina,&Wang, Fei-Yue.(2022).HMDRL: Hierarchical Mixed Deep Reinforcement Learning to Balance Vehicle Supply and Demand.IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,12.
MLA Xi, Jinhao,et al."HMDRL: Hierarchical Mixed Deep Reinforcement Learning to Balance Vehicle Supply and Demand".IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS (2022):12.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Xi, Jinhao]'s Articles
[Zhu, Fenghua]'s Articles
[Ye, Peijun]'s Articles
Baidu academic
Similar articles in Baidu academic
[Xi, Jinhao]'s Articles
[Zhu, Fenghua]'s Articles
[Ye, Peijun]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Xi, Jinhao]'s Articles
[Zhu, Fenghua]'s Articles
[Ye, Peijun]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.