Error Bound Analysis of Q-Function for Discounted Optimal Control Problems With Policy Iteration

doi:10.1109/TSMC.2016.2563982

CASIA OpenIR > 多模态人工智能系统全国重点实验室 > 复杂系统智能机理与平行控制团队

	Error Bound Analysis of Q-Function for Discounted Optimal Control Problems With Policy Iteration
	Yan, Pengfei1 ; Wang, Ding1 ; Li, Hongliang 2; Liu, Derong 3
发表期刊	IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
	2017-07-01
卷号	47 期号:7 页码:1207-1216
文章类型	Article
摘要	In this paper, we present error bound analysis of the Q-function for the action-dependent adaptive dynamic programming for solving discounted optimal control problems of unknown discrete-time nonlinear systems. The convergence of Q-functions derived by a policy iteration algorithm under ideal conditions is given. Considering the approximated errors of the Q-function and control policy in the policy evaluation step and policy improvement step, we establish error bounds of approximate Q-functions in each iteration. With the given boundedness conditions, the approximate Q-function will converge to a finite neighborhood of the optimal Q-function. To implement the presented algorithm, two three-layer neural networks are employed to approximate the Q-function and the control policy, respectively. Finally, a simulation example is utilized to verify the validity of the presented algorithm.
关键词	Adaptive Dynamic Programming (Adp) Error Analysis Nonlinear Systems Policy Iteration Q-function
WOS标题词	Science & Technology ; Technology
DOI	10.1109/TSMC.2016.2563982
关键词[WOS]	TIME NONLINEAR-SYSTEMS ; APPROXIMATE VALUE-ITERATION ; UNKNOWN INTERNAL DYNAMICS ; ADAPTIVE OPTIMAL-CONTROL ; OPTIMAL-CONTROL DESIGN ; H-INFINITY CONTROL ; ZERO-SUM GAMES ; INPUT CONSTRAINTS ; HJB SOLUTION ; REINFORCEMENT
收录类别	SCI
语种	英语
项目资助者	National Natural Science Foundation of China(61233001 ; Beijing Natural Science Foundation(4162065) ; Early Career Development Award of SKLMCCS ; 61273140 ; 61304086 ; 61374105 ; 61533017 ; U1501251)
WOS研究方向	Automation & Control Systems ; Computer Science
WOS类目	Automation & Control Systems ; Computer Science, Cybernetics
WOS记录号	WOS:000404354600014
引用统计	被引频次：23[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/15223
专题	多模态人工智能系统全国重点实验室_复杂系统智能机理与平行控制团队
作者单位	1.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China 2.IBM Res China, Beijing 100193, Peoples R China 3.Univ Sci & Technol Beijing, Sch Automat & Elect Engn, Beijing 100083, Peoples R China
第一作者单位	中国科学院自动化研究所
推荐引用方式 GB/T 7714	Yan, Pengfei,Wang, Ding,Li, Hongliang,et al. Error Bound Analysis of Q-Function for Discounted Optimal Control Problems With Policy Iteration[J]. IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS,2017,47(7):1207-1216.
APA	Yan, Pengfei,Wang, Ding,Li, Hongliang,&Liu, Derong.(2017).Error Bound Analysis of Q-Function for Discounted Optimal Control Problems With Policy Iteration.IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS,47(7),1207-1216.
MLA	Yan, Pengfei,et al."Error Bound Analysis of Q-Function for Discounted Optimal Control Problems With Policy Iteration".IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS 47.7(2017):1207-1216.

条目包含的文件		下载所有文件
文件名称/大小	文献类型	版本类型	开放类型	使用许可
[00-J-2017-TSMC] Err（625KB）	期刊论文	作者接受稿	开放获取	CC BY-NC-SA	浏览下载