Data-efficient model-based reinforcement learning with trajectory discrimination

doi:10.1007/s40747-023-01247-5

	Data-efficient model-based reinforcement learning with trajectory discrimination
	Qu, Tuo 1; Duan, Fuqing 1; Zhang, Junge2 ; Zhao, Bo3 ; Huang, Wenzhen2
发表期刊	COMPLEX & INTELLIGENT SYSTEMS
ISSN	2199-4536
	2023-10-11
页码	10
通讯作者	Duan, Fuqing(fqduan@bnu.edu.cn)
摘要	Deep reinforcement learning has always been used to solve high-dimensional complex sequential decision problems. However, one of the biggest challenges for reinforcement learning is sample efficiency, especially for high-dimensional complex problems. Model-based reinforcement learning can solve the problem with a learned world model, but the performance is limited by the imperfect world model, so it usually has worse approximate performance than model-free reinforcement learning. In this paper, we propose a novel model-based reinforcement learning algorithm called World Model with Trajectory Discrimination (WMTD). We learn the representation of temporal dynamics information by adding a trajectory discriminator to the world model, and then compute the weight of state value estimation based on the trajectory discriminator to optimize the policy. Specifically, we augment the trajectories to generate negative samples and train a trajectory discriminator that shares the feature extractor with the world model. Experimental results demonstrate that our method improves the sample efficiency and achieves state-of-the-art performance on DeepMind control tasks.
关键词	Reinforcement learning Deep learning Continuous control task World model
DOI	10.1007/s40747-023-01247-5
收录类别	SCI
语种	英语
资助项目	This work is supported by National Key Research and Development Project Grant, grant/award number: 2018AAA01008-02.[2018AAA01008-02] ; National Key Research and Development Project Grant
项目资助者	This work is supported by National Key Research and Development Project Grant, grant/award number: 2018AAA01008-02. ; National Key Research and Development Project Grant
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence
WOS记录号	WOS:001081013100001
出版者	SPRINGER HEIDELBERG
引用统计
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/53009
专题	复杂系统认知与决策实验室
通讯作者	Duan, Fuqing
作者单位	1.Beijing Normal Univ, Sch Artificial Intelligence, 19 Xinjiekou Outer St, Beijing 100875, Peoples R China 2.Chinese Acad Sci, Inst Automat, 95 Zhongguancun East Rd, Beijing 100190, Peoples R China 3.Beijing Normal Univ, Sch Syst Sci, 19 Xinjiekou Outer St, Beijing 100875, Peoples R China
推荐引用方式 GB/T 7714	Qu, Tuo,Duan, Fuqing,Zhang, Junge,et al. Data-efficient model-based reinforcement learning with trajectory discrimination[J]. COMPLEX & INTELLIGENT SYSTEMS,2023:10.
APA	Qu, Tuo,Duan, Fuqing,Zhang, Junge,Zhao, Bo,&Huang, Wenzhen.(2023).Data-efficient model-based reinforcement learning with trajectory discrimination.COMPLEX & INTELLIGENT SYSTEMS,10.
MLA	Qu, Tuo,et al."Data-efficient model-based reinforcement learning with trajectory discrimination".COMPLEX & INTELLIGENT SYSTEMS (2023):10.