| HMDRL: Hierarchical Mixed Deep Reinforcement Learning to Balance Vehicle Supply and Demand | |
Xi, Jinhao1,2; Zhu, Fenghua1,2 ; Ye, Peijun1,2; Lv, Yisheng1,2; Tang, Haina2; Wang, Fei-Yue1,2
| |
| Source Publication | IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
 |
| ISSN | 1524-9050 |
| 2022-07-25 | |
| Pages | 12 |
| Corresponding Author | Zhu, Fenghua(fenghua.zhu@ia.ac.cn) ; Ye, Peijun(peijun_ye@hotmail.com) |
| Abstract | The imbalance of vehicle supply and demand is a common phenomenon that influences the efficiency of online ride-hailing systems greatly. To address this problem, a three-level hierarchical mixed deep reinforcement learning method (HMDRL) is proposed to reposition idle vehicles. A manager operates at the top level, where action-abstraction is conducted from the time dimension and is adaptive for spatially scalable and time-varying systems. Coordinators locate at the middle level and a parallel coordination mechanism that is independent of the decision order is designed to improve the efficiency of the repositioning. The bottom level is composed of executive workers to reposition vehicles with mixed states and the states contain spatiotemporal information of agents' neighbor areas. Two reward functions are designed for the manager and the coordinators, respectively, aiming to improve the training effect by avoiding sparse rewards. A simulator based on real orders is designed and HMDRL is compared with six methods. Experimental results demonstrate that HMDRL outperforms all the other methods. In three comparison experiments, the order response rate is increased by 0.62% to 8.29%, 1.5% to 7.78%, 1.18% to 4.75%, respectively. |
| Keyword | Reinforcement learning Pricing Heuristic algorithms Supply and demand Surges Vehicle dynamics Vehicles Deep reinforcement learning online ride-hailing system hierarchical repositioning framework parallel coordination mechanism mixed state |
| DOI | 10.1109/TITS.2022.3191752 |
| Indexed By | SCI |
| Language | 英语 |
| Funding Project | Key-Area Research and Development Program of Guangdong Province[2020B0909050001] ; National Natural Science Foundation of China (NSFC)[U1811463] ; National Natural Science Foundation of China (NSFC)[U1909204] ; National Natural Science Foundation of China (NSFC)[61876011] ; National Natural Science Foundation of China (NSFC)[52071312] ; Chinese Academy of Sciences (CAS) Science and Technology Service Network Plan (STS) Dongguan Project[20201600200132] |
| Funding Organization | Key-Area Research and Development Program of Guangdong Province ; National Natural Science Foundation of China (NSFC) ; Chinese Academy of Sciences (CAS) Science and Technology Service Network Plan (STS) Dongguan Project |
| WOS Research Area | Engineering ; Transportation |
| WOS Subject | Engineering, Civil ; Engineering, Electrical & Electronic ; Transportation Science & Technology |
| WOS ID | WOS:000833052300001 |
| Publisher | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| Citation statistics | |
| Document Type | 期刊论文 |
| Identifier | http://ir.ia.ac.cn/handle/173211/49762 |
| Collection | 复杂系统管理与控制国家重点实验室_平行智能技术与系统团队 |
| Corresponding Author | Zhu, Fenghua; Ye, Peijun |
| Affiliation | 1.Chinese Acad Sci, State Key Lab Management & Control Complex Syst, Inst Automat, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China |
| First Author Affilication | Institute of Automation, Chinese Academy of Sciences |
| Corresponding Author Affilication | Institute of Automation, Chinese Academy of Sciences |
| Recommended Citation GB/T 7714 | Xi, Jinhao,Zhu, Fenghua,Ye, Peijun,et al. HMDRL: Hierarchical Mixed Deep Reinforcement Learning to Balance Vehicle Supply and Demand[J]. IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,2022:12. |
| APA | Xi, Jinhao,Zhu, Fenghua,Ye, Peijun,Lv, Yisheng,Tang, Haina,&Wang, Fei-Yue.(2022).HMDRL: Hierarchical Mixed Deep Reinforcement Learning to Balance Vehicle Supply and Demand.IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,12. |
| MLA | Xi, Jinhao,et al."HMDRL: Hierarchical Mixed Deep Reinforcement Learning to Balance Vehicle Supply and Demand".IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS (2022):12. |
| Files in This Item: | There are no files associated with this item. | |||||
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment