Network-Wide Traffic Signal Control Based on MARL With Hierarchical Nash-Stackelberg Game Model

doi:10.1109/ACCESS.2023.3345448

	Network-Wide Traffic Signal Control Based on MARL With Hierarchical Nash-Stackelberg Game Model
	Shen, Hui 1,2; Zhao, Hongxia3 ; Zhang, Zundong 4; Yang, Xun 4; Song, Yutong 4; Liu, Xiaoming 4
发表期刊	IEEE ACCESS
ISSN	2169-3536
	2023
卷号	11 页码:145085-145100
通讯作者	Zhang, Zundong(zdzhang@ncut.edu.cn)
摘要	Network-wide traffic signal control is an important means of relieving urban congestion, reducing traffic accidents, and improving traffic efficiency. However, solving the problem of computational complexity caused by multi-intersection games is challenging. To address this issue, we propose a Nash-Stackelberg hierarchical game model that considers the importance of different intersections in the road network and the game relationships between intersections. The model takes into account traffic control strategies between and within sub-areas of the road network, with important intersections in the two sub-areas as the game subject at the upper layer and secondary intersections as the game subject at the lower layer. Furthermore, we propose two reinforcement learning algorithms (NSHG-QL and NSHG-DQN) based on the Nash-Stackelberg hierarchical game model to realize coordinated control of traffic signals in urban areas. Experimental results show that, compared to basic game model solving algorithms, NSHG-QL and NSHG-DQN algorithms can reduce the average travel time and time loss of vehicles at intersections, increase average speed and road occupancy, and coordinate secondary intersections to make optimal strategy selections based on satisfying the upper-layer game between important intersections. Moreover, the multi-agent reinforcement learning algorithms based on this hierarchical game model can significantly improve learning performance and convergence.
关键词	Games Roads Approximation algorithms Q-learning Multi-agent systems Process control Optimization Reinforcement learning Traffic control Network-wide traffic signal control hierarchical game model multi-agent reinforcement learning
DOI	10.1109/ACCESS.2023.3345448
关键词[WOS]	MULTIAGENT SYSTEMS ; REINFORCEMENT
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation Project
项目资助者	National Natural Science Foundation Project
WOS研究方向	Computer Science ; Engineering ; Telecommunications
WOS类目	Computer Science, Information Systems ; Engineering, Electrical & Electronic ; Telecommunications
WOS记录号	WOS:001132233900001
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/54911
专题	多模态人工智能系统全国重点实验室
通讯作者	Zhang, Zundong
作者单位	1.North China Univ Technol, Sch Elect & Control Engn, Beijing 100037, Peoples R China 2.Beijing Municipal Traff Management Bur, Beijing 100037, Peoples R China 3.Chinese Acad Sci, Inst Automat, State Key Lab Multimodal Artificial Intelligence S, Beijing 100190, Peoples R China 4.North China Univ Technol, Beijing Key Lab Urban Rd Traff Intelligent Technol, Beijing 100144, Peoples R China
推荐引用方式 GB/T 7714	Shen, Hui,Zhao, Hongxia,Zhang, Zundong,et al. Network-Wide Traffic Signal Control Based on MARL With Hierarchical Nash-Stackelberg Game Model[J]. IEEE ACCESS,2023,11:145085-145100.
APA	Shen, Hui,Zhao, Hongxia,Zhang, Zundong,Yang, Xun,Song, Yutong,&Liu, Xiaoming.(2023).Network-Wide Traffic Signal Control Based on MARL With Hierarchical Nash-Stackelberg Game Model.IEEE ACCESS,11,145085-145100.
MLA	Shen, Hui,et al."Network-Wide Traffic Signal Control Based on MARL With Hierarchical Nash-Stackelberg Game Model".IEEE ACCESS 11(2023):145085-145100.