Counterfactual Evolutionary Reasoning for Virtual Driver Reinforcement Learning in Safe Driving

doi:10.1109/TIV.2023.3322694

	Counterfactual Evolutionary Reasoning for Virtual Driver Reinforcement Learning in Safe Driving
	Ye, Peijun1 ; Qi, Hao 2; Zhu, Fenghua1 ; Lv, Yisheng1
发表期刊	IEEE TRANSACTIONS ON INTELLIGENT VEHICLES
ISSN	2379-8858
	2023-12-01
卷号	8 期号:12 页码:4696-4705
通讯作者	Ye, Peijun(peijun_ye@hotmail.com)
摘要	Safety is the primary concern in the motion planning and decision-making of the virtual driver that provides prescriptions to the real human driver and even performs self-driving in the absence of human take-over. For such an issue, traditional reinforcement learning methods, limited by their learning mechanisms, suffer from a slow convergence of model training as well as a less consideration for early warning of possible accidents. To address the above deficiency, this paper proposes a new method based on counterfactual evolutionary reasoning that can be used to build the virtual driver. The method treats safe driving as a sequential decision-making problem with sparse rewards, and employs counterfactual evolutionary reasoning to guide the searching direction as well as to accelerate the model training. An intervention mechanism from outlier distributions is further introduced to enhance the model's ability of exploration. Experiments in the virtual test environment indicate that the proposed method, compared with other typical reinforcement learning techniques, both achieves a higher safe arrival rate and a faster convergence speed.
关键词	Safe driving reinforcement learning counter factual reasoning evolutionary search
DOI	10.1109/TIV.2023.3322694
关键词[WOS]	VEHICLES
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China
项目资助者	National Natural Science Foundation of China
WOS研究方向	Computer Science ; Engineering ; Transportation
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic ; Transportation Science & Technology
WOS记录号	WOS:001140418700007
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/55471
专题	多模态人工智能系统全国重点实验室
通讯作者	Ye, Peijun
作者单位	1.Chinese Acad Sci, Inst Automat, State Key Lab Multimodal Artificial Intelligence, Beijing 100190, Peoples R China 2.Shandong Jiaotong Univ, Sch Rail Transportat, Jinan 250357, Peoples R China
第一作者单位	中国科学院自动化研究所
通讯作者单位	中国科学院自动化研究所
推荐引用方式 GB/T 7714	Ye, Peijun,Qi, Hao,Zhu, Fenghua,et al. Counterfactual Evolutionary Reasoning for Virtual Driver Reinforcement Learning in Safe Driving[J]. IEEE TRANSACTIONS ON INTELLIGENT VEHICLES,2023,8(12):4696-4705.
APA	Ye, Peijun,Qi, Hao,Zhu, Fenghua,&Lv, Yisheng.(2023).Counterfactual Evolutionary Reasoning for Virtual Driver Reinforcement Learning in Safe Driving.IEEE TRANSACTIONS ON INTELLIGENT VEHICLES,8(12),4696-4705.
MLA	Ye, Peijun,et al."Counterfactual Evolutionary Reasoning for Virtual Driver Reinforcement Learning in Safe Driving".IEEE TRANSACTIONS ON INTELLIGENT VEHICLES 8.12(2023):4696-4705.