Knowledge Commons of Institute of Automation,CAS
HMDRL: Hierarchical Mixed Deep Reinforcement Learning to Balance Vehicle Supply and Demand | |
Xi, Jinhao1,2; Zhu, Fenghua1,2; Ye, Peijun1,2; Lv, Yisheng1,2; Tang, Haina2; Wang, Fei-Yue1,2 | |
发表期刊 | IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS |
ISSN | 1524-9050 |
2022-07-25 | |
页码 | 12 |
摘要 | The imbalance of vehicle supply and demand is a common phenomenon that influences the efficiency of online ride-hailing systems greatly. To address this problem, a three-level hierarchical mixed deep reinforcement learning method (HMDRL) is proposed to reposition idle vehicles. A manager operates at the top level, where action-abstraction is conducted from the time dimension and is adaptive for spatially scalable and time-varying systems. Coordinators locate at the middle level and a parallel coordination mechanism that is independent of the decision order is designed to improve the efficiency of the repositioning. The bottom level is composed of executive workers to reposition vehicles with mixed states and the states contain spatiotemporal information of agents' neighbor areas. Two reward functions are designed for the manager and the coordinators, respectively, aiming to improve the training effect by avoiding sparse rewards. A simulator based on real orders is designed and HMDRL is compared with six methods. Experimental results demonstrate that HMDRL outperforms all the other methods. In three comparison experiments, the order response rate is increased by 0.62% to 8.29%, 1.5% to 7.78%, 1.18% to 4.75%, respectively. |
关键词 | deep reinforcement learning online ride-hailing system hierarchical repositioning framework parallel coordination mechanism mixed state |
DOI | 10.1109/TITS.2022.3191752 |
收录类别 | SCI |
语种 | 英语 |
资助项目 | Key-Area Research and Development Program of Guangdong Province[2020B0909050001] ; National Natural Science Foundation of China (NSFC)[U1811463] ; National Natural Science Foundation of China (NSFC)[U1909204] ; National Natural Science Foundation of China (NSFC)[61876011] ; National Natural Science Foundation of China (NSFC)[52071312] ; Chinese Academy of Sciences (CAS) Science and Technology Service Network Plan (STS) Dongguan Project[20201600200132] |
项目资助者 | Key-Area Research and Development Program of Guangdong Province ; National Natural Science Foundation of China (NSFC) ; Chinese Academy of Sciences (CAS) Science and Technology Service Network Plan (STS) Dongguan Project |
WOS研究方向 | Engineering ; Transportation |
WOS类目 | Engineering, Civil ; Engineering, Electrical & Electronic ; Transportation Science & Technology |
WOS记录号 | WOS:000833052300001 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
是否为代表性论文 | 是 |
七大方向——子方向分类 | 人工智能+交通 |
国重实验室规划方向分类 | 多智能体决策 |
是否有论文关联数据集需要存交 | 否 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/49762 |
专题 | 多模态人工智能系统全国重点实验室_平行智能技术与系统团队 |
通讯作者 | Zhu, Fenghua; Ye, Peijun |
作者单位 | 1.The State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China 2.The School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Xi, Jinhao,Zhu, Fenghua,Ye, Peijun,et al. HMDRL: Hierarchical Mixed Deep Reinforcement Learning to Balance Vehicle Supply and Demand[J]. IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,2022:12. |
APA | Xi, Jinhao,Zhu, Fenghua,Ye, Peijun,Lv, Yisheng,Tang, Haina,&Wang, Fei-Yue.(2022).HMDRL: Hierarchical Mixed Deep Reinforcement Learning to Balance Vehicle Supply and Demand.IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,12. |
MLA | Xi, Jinhao,et al."HMDRL: Hierarchical Mixed Deep Reinforcement Learning to Balance Vehicle Supply and Demand".IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS (2022):12. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
HMDRL_Hierarchical_M(3316KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论