CASIA OpenIR  > 复杂系统管理与控制国家重点实验室  > 平行控制
Multi-step heuristic dynamic programming for optimal control of nonlinear discrete-time systems
Luo, Biao1; Liu, Derong2; Huang, Tingwen4; Yang, Xiong3; Ma, Hongwen1
Source PublicationINFORMATION SCIENCES
2017-10-01
Volume411Issue:0Pages:66-83
SubtypeArticle
Abstract

Policy iteration and value iteration are two main iterative adaptive dynamic programming frameworks for solving optimal control problems. Policy iteration converges fast while requiring an initial stabilizing control policy, which is a strict constraint in practice. Value iteration avoids the requirement of initial admissible control policy while converging much slowly. This paper tries to utilize the advantages of policy iteration and value iteration, and avoids their drawbacks at the same time. Therefore, a multi-step heuristic dynamic programming (MsHDP) method is developed for solving the optimal control problem of nonlinear discrete-time systems. MsHDP speeds up value iteration and avoids the requirement of initial admissible control policy in policy iteration at the same time. The convergence theory of MsHDP is established by proving that it converges to the solution of the Bellman equation. For implementation purpose, the actor-critic neural network (NN) structure is developed. The critic NN is employed to estimate the value function and its NN weight vector is computed with a least-square scheme. The actor NN is used to estimate the control policy and a gradient descent method is proposed for updating its NN weight vector. According to the comparative simulation studies on two examples, the effectiveness and advantages of MsHDP are verified. (C) 2017 Elsevier Inc. All rights reserved.

KeywordOptimal Control Multi-step Heuristic Dynamic Programming Adaptive Dynamic Programming Nonlinear Systems Discrete-time Neural Networks
WOS HeadingsScience & Technology ; Technology
DOI10.1016/j.ins.2017.05.005
WOS KeywordSpatially Distributed Processes ; Optimal Tracking Control ; Horizon Optimal-control ; Neural-network Control ; Optimal-control Scheme ; H-infinity Control ; Policy Iteration ; Control Design ; Feedback-control ; Algorithm
Indexed BySCI
Language英语
Funding OrganizationNational Natural Science Foundation of China(61533017 ; Early Career Development Award of SKLMCCS ; NPRP from the Qatar National Research Fund (a member of Qatar Foundation)(NPRP 9 166-1-031) ; U1501251 ; 61374105 ; 61503377 ; 61233001)
WOS Research AreaComputer Science
WOS SubjectComputer Science, Information Systems
WOS IDWOS:000404197200005
Citation statistics
Cited Times:6[WOS]   [WOS Record]     [Related Records in WOS]
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/15245
Collection复杂系统管理与控制国家重点实验室_平行控制
Affiliation1.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China
2.Guangdong Univ Technol, Sch Automat, Guangzhou 510006, Guangdong, Peoples R China
3.Tianjin Univ, Sch Elect & Informat Engn, Tianjin 300072, Peoples R China
4.Texas A&M Univ Qatar, POB 23874, Doha, Qatar
Recommended Citation
GB/T 7714
Luo, Biao,Liu, Derong,Huang, Tingwen,et al. Multi-step heuristic dynamic programming for optimal control of nonlinear discrete-time systems[J]. INFORMATION SCIENCES,2017,411(0):66-83.
APA Luo, Biao,Liu, Derong,Huang, Tingwen,Yang, Xiong,&Ma, Hongwen.(2017).Multi-step heuristic dynamic programming for optimal control of nonlinear discrete-time systems.INFORMATION SCIENCES,411(0),66-83.
MLA Luo, Biao,et al."Multi-step heuristic dynamic programming for optimal control of nonlinear discrete-time systems".INFORMATION SCIENCES 411.0(2017):66-83.
Files in This Item: Download All
File Name/Size DocType Version Access License
2017-10-INS-Multi-st(1092KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Luo, Biao]'s Articles
[Liu, Derong]'s Articles
[Huang, Tingwen]'s Articles
Baidu academic
Similar articles in Baidu academic
[Luo, Biao]'s Articles
[Liu, Derong]'s Articles
[Huang, Tingwen]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Luo, Biao]'s Articles
[Liu, Derong]'s Articles
[Huang, Tingwen]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: 2017-10-INS-Multi-step heuristic dynamic programming for optimal control of nonlinear discrete-time systems.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.