Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design
Luo, Biao1; Wu, Huai-Ning2; Huang, Tingwen3; Liu, Derong1
发表期刊AUTOMATICA
2014-12-01
卷号50期号:12页码:3281-3290
文章类型Article
摘要This paper addresses the model-free nonlinear optimal control problem based on data by introducing the reinforcement learning (RL) technique. It is known that the nonlinear optimal control problem relies on the solution of the Hamilton-Jacobi-Bellman (HJB) equation, which is a nonlinear partial differential equation that is generally impossible to be solved analytically. Even worse, most practical systems are too complicated to establish an accurate mathematical model. To overcome these difficulties, we propose a data-based approximate policy iteration (API) method by using real system data rather than a system model. Firstly, a model-free policy iteration algorithm is derived and its convergence is proved. The implementation of the algorithm is based on the actor-critic structure, where actor and critic neural networks (NNs) are employed to approximate the control policy and cost function, respectively. To update the weights of actor and critic NNs, a least-square approach is developed based on the method of weighted residuals. The data-based API is an off-policy RL method, where the "exploration" is improved by arbitrarily sampling data on the state and input domain. Finally, we test the data-based API control design method on a simple nonlinear system, and further apply it to a rotational/translational actuator system. The simulation results demonstrate the effectiveness of the proposed method. (C) 2014 Elsevier Ltd. All rights reserved.
关键词Nonlinear Optimal Control Reinforcement Learning Off-policy Data-based Approximate Policy Iteration Neural Network Hamilton-jacobi-bellman Equation
WOS标题词Science & Technology ; Technology
关键词[WOS]ADAPTIVE OPTIMAL-CONTROL ; LINEAR-SYSTEMS ; REINFORCEMENT ; STABILIZATION ; EQUATION
收录类别SCI
语种英语
WOS研究方向Automation & Control Systems ; Engineering
WOS类目Automation & Control Systems ; Engineering, Electrical & Electronic
WOS记录号WOS:000347760100036
引用统计
被引频次:208[WOS]   [WOS记录]     [WOS相关记录]
文献类型期刊论文
条目标识符http://ir.ia.ac.cn/handle/173211/3824
专题多模态人工智能系统全国重点实验室_复杂系统智能机理与平行控制团队
通讯作者Wu, Huai-Ning
作者单位1.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China
2.Beijing Univ Aeronaut & Astronaut, Beihang Univ, Sci & Technol Aircraft Control Lab, Beijing 100191, Peoples R China
3.Texas A&M Univ Qatar, Doha, Qatar
第一作者单位中国科学院自动化研究所
推荐引用方式
GB/T 7714
Luo, Biao,Wu, Huai-Ning,Huang, Tingwen,et al. Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design[J]. AUTOMATICA,2014,50(12):3281-3290.
APA Luo, Biao,Wu, Huai-Ning,Huang, Tingwen,&Liu, Derong.(2014).Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design.AUTOMATICA,50(12),3281-3290.
MLA Luo, Biao,et al."Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design".AUTOMATICA 50.12(2014):3281-3290.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
2014Automatica_Data-(668KB)期刊论文作者接受稿开放获取CC BY-NC-SA浏览
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[Luo, Biao]的文章
[Wu, Huai-Ning]的文章
[Huang, Tingwen]的文章
百度学术
百度学术中相似的文章
[Luo, Biao]的文章
[Wu, Huai-Ning]的文章
[Huang, Tingwen]的文章
必应学术
必应学术中相似的文章
[Luo, Biao]的文章
[Wu, Huai-Ning]的文章
[Huang, Tingwen]的文章
相关权益政策
暂无数据
收藏/分享
文件名: 2014Automatica_Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design.pdf
格式: Adobe PDF
此文件暂不支持浏览
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。