Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design

CASIA OpenIR > 多模态人工智能系统全国重点实验室 > 复杂系统智能机理与平行控制团队

	Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design
	Luo, Biao1 ; Wu, Huai-Ning 2; Huang, Tingwen 3; Liu, Derong 1
发表期刊	AUTOMATICA
	2014-12-01
卷号	50 期号:12 页码:3281-3290
文章类型	Article
摘要	This paper addresses the model-free nonlinear optimal control problem based on data by introducing the reinforcement learning (RL) technique. It is known that the nonlinear optimal control problem relies on the solution of the Hamilton-Jacobi-Bellman (HJB) equation, which is a nonlinear partial differential equation that is generally impossible to be solved analytically. Even worse, most practical systems are too complicated to establish an accurate mathematical model. To overcome these difficulties, we propose a data-based approximate policy iteration (API) method by using real system data rather than a system model. Firstly, a model-free policy iteration algorithm is derived and its convergence is proved. The implementation of the algorithm is based on the actor-critic structure, where actor and critic neural networks (NNs) are employed to approximate the control policy and cost function, respectively. To update the weights of actor and critic NNs, a least-square approach is developed based on the method of weighted residuals. The data-based API is an off-policy RL method, where the "exploration" is improved by arbitrarily sampling data on the state and input domain. Finally, we test the data-based API control design method on a simple nonlinear system, and further apply it to a rotational/translational actuator system. The simulation results demonstrate the effectiveness of the proposed method. (C) 2014 Elsevier Ltd. All rights reserved.
关键词	Nonlinear Optimal Control Reinforcement Learning Off-policy Data-based Approximate Policy Iteration Neural Network Hamilton-jacobi-bellman Equation
WOS标题词	Science & Technology ; Technology
关键词[WOS]	ADAPTIVE OPTIMAL-CONTROL ; LINEAR-SYSTEMS ; REINFORCEMENT ; STABILIZATION ; EQUATION
收录类别	SCI
语种	英语
WOS研究方向	Automation & Control Systems ; Engineering
WOS类目	Automation & Control Systems ; Engineering, Electrical & Electronic
WOS记录号	WOS:000347760100036
引用统计	被引频次：209[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/3824
专题	多模态人工智能系统全国重点实验室_复杂系统智能机理与平行控制团队
通讯作者	Wu, Huai-Ning
作者单位	1.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China 2.Beijing Univ Aeronaut & Astronaut, Beihang Univ, Sci & Technol Aircraft Control Lab, Beijing 100191, Peoples R China 3.Texas A&M Univ Qatar, Doha, Qatar
第一作者单位	中国科学院自动化研究所
推荐引用方式 GB/T 7714	Luo, Biao,Wu, Huai-Ning,Huang, Tingwen,et al. Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design[J]. AUTOMATICA,2014,50(12):3281-3290.
APA	Luo, Biao,Wu, Huai-Ning,Huang, Tingwen,&Liu, Derong.(2014).Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design.AUTOMATICA,50(12),3281-3290.
MLA	Luo, Biao,et al."Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design".AUTOMATICA 50.12(2014):3281-3290.

条目包含的文件
文件名称/大小	文献类型	版本类型	开放类型	使用许可
2014Automatica_Data-（668KB）	期刊论文	作者接受稿	开放获取	CC BY-NC-SA	浏览