Event-Triggered Deep Reinforcement Learning Using Parallel Control: A Case Study in Autonomous Driving

doi:10.1109/TIV.2023.3262132

	Event-Triggered Deep Reinforcement Learning Using Parallel Control: A Case Study in Autonomous Driving
	Lu, Jingwei1,2 ; Han, Liyuan3,4 ; Wei, Qinglai3,4 ; Wang, Xiao5,6 ; Dai, Xingyuan3,4 ; Wang, Fei-Yue3,4
发表期刊	IEEE TRANSACTIONS ON INTELLIGENT VEHICLES
ISSN	2379-8858
	2023-04-01
卷号	8 期号:4 页码:2821-2831
通讯作者	Lu, Jingwei(lujingweihh@gmail.com)
摘要	This paper utilizes parallel control to investigate the problem of event-triggered deep reinforcement learning and develops an event-triggered deep Q-network (ETDQN) for decision-making of autonomous driving, without training an explicit triggering condition. Based on the framework of parallel control, the developed ETDQN incorporates information of actions into the feedback and constructs a dynamic control policy. First, in the realization of the dynamic control policy, we integrate the current state and the previous action to construct the augmented state as well as the augmented Markov decision process. Meanwhile, it is shown theoretically that the goal of the developed dynamic control policy is to learn the variation rate of the action. The augmented state contains information of the current state and the previous action, which enables the developed ETDQN to directly design the immediate reward considering communication loss. Then, based on dueling double deep Q-network (dueling DDQN), we establish the augmented action-value, value, and advantage functions to directly learn the optimal event-triggered decision-making policy of autonomous driving without an explicit triggering condition. It is worth noticing that the developed ETDQN applies to various deep Q-networks (DQNs). Empirical results demonstrate that, in event-triggered control, the developed ETDQN outperforms dueling DDQN and reduces communication loss effectively.
关键词	Autonomous vehicles Decision making Path planning Training Optimal control Deep learning Complex systems Autonomous driving deep reinforcement learning deep Q-network event-triggered control parallel control
DOI	10.1109/TIV.2023.3262132
收录类别	SCI
语种	英语
资助项目	Key Research and Development Program 2020 of Guangzhou[202007050002] ; Key-Area Research and Development Program of Guangdong Province[U1811463] ; National Natural Science Foundation of China[2020B090921003] ; Motion G, Inc. Collaborative Research Project for Modeling, Decision and Control Algorithms of Servo Drive Systems
项目资助者	Key Research and Development Program 2020 of Guangzhou ; Key-Area Research and Development Program of Guangdong Province ; National Natural Science Foundation of China ; Motion G, Inc. Collaborative Research Project for Modeling, Decision and Control Algorithms of Servo Drive Systems
WOS研究方向	Computer Science ; Engineering ; Transportation
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic ; Transportation Science & Technology
WOS记录号	WOS:000994739000019
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：25[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/53418
专题	多模态人工智能系统全国重点实验室
通讯作者	Lu, Jingwei
作者单位	1.Qingdao Acad Intelligent Ind, Parallel Intelligence Innovat Res Ctr, Qingdao 266109, Peoples R China 2.Chinese Acad Sci, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China 3.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China 4.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China 5.Anhui Univ, Sch Artificial Intelligence, Hefei 230031, Peoples R China 6.Qingdao Acad Intelligent Ind, Qingdao 266114, Peoples R China
第一作者单位	中国科学院自动化研究所
通讯作者单位	中国科学院自动化研究所
推荐引用方式 GB/T 7714	Lu, Jingwei,Han, Liyuan,Wei, Qinglai,et al. Event-Triggered Deep Reinforcement Learning Using Parallel Control: A Case Study in Autonomous Driving[J]. IEEE TRANSACTIONS ON INTELLIGENT VEHICLES,2023,8(4):2821-2831.
APA	Lu, Jingwei,Han, Liyuan,Wei, Qinglai,Wang, Xiao,Dai, Xingyuan,&Wang, Fei-Yue.(2023).Event-Triggered Deep Reinforcement Learning Using Parallel Control: A Case Study in Autonomous Driving.IEEE TRANSACTIONS ON INTELLIGENT VEHICLES,8(4),2821-2831.
MLA	Lu, Jingwei,et al."Event-Triggered Deep Reinforcement Learning Using Parallel Control: A Case Study in Autonomous Driving".IEEE TRANSACTIONS ON INTELLIGENT VEHICLES 8.4(2023):2821-2831.