CASIA OpenIR  > 复杂系统管理与控制国家重点实验室  > 平行控制
Model-Free Optimal Tracking Control via Critic-Only Q-Learning
Luo, Biao1; Liu, Derong2; Huang, Tingwen3; Wang, Ding1; Luo,Biao
Source PublicationIEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
2016-10-01
Volume27Issue:10Pages:2134-2144
SubtypeArticle
AbstractModel-free control is an important and promising topic in control fields, which has attracted extensive attention in the past few years. In this paper, we aim to solve the model-free optimal tracking control problem of nonaffine non-linear discrete-time systems. A critic-only Q-learning (CoQL) method is developed, which learns the optimal tracking control from real system data, and thus avoids solving the tracking Hamilton-Jacobi-Bellman equation. First, the Q-learning algorithm is proposed based on the augmented system, and its convergence is established. Using only one neural network for approximating the Q-function, the CoQL method is developed to implement the Q-learning algorithm. Furthermore, the convergence of the CoQL method is proved with the consideration of neural network approximation error. With the convergent Q-function obtained from the CoQL method, the adaptive optimal tracking control is designed based on the gradient descent scheme. Finally, the effectiveness of the developed CoQL method is demonstrated through simulation studies. The developed CoQL method learns with off-policy data and implements with a critic-only structure, thus it is easy to realize and overcome the inadequate exploration problem.
Other Abstract
KeywordCritic-only Q-learning (Coql) Model-free Nonaffine Nonlinear Systems Optimal Tracking Control
WOS HeadingsScience & Technology ; Technology
DOI10.1109/TNNLS.2016.2585520
WOS KeywordTIME NONLINEAR-SYSTEMS ; H-INFINITY CONTROL ; ADAPTIVE OPTIMAL-CONTROL ; SPATIALLY DISTRIBUTED PROCESSES ; LINEAR-SYSTEMS ; CONTROL DESIGN ; UNKNOWN DYNAMICS ; CONTROL SCHEME ; ATTITUDE TRACKING ; POLICY ITERATION
Indexed BySCI
Language英语
Funding OrganizationNational Natural Science Foundation of China(61233001 ; State Key Laboratory of Management and Control for Complex Systems ; National Priorities Research Program through the Qatar National Research Fund (a member of Qatar Foundation)(NPRP 7-1482-1-278) ; 61273140 ; 61304086 ; 61374105 ; 61503377 ; 61533017 ; U1501251)
WOS Research AreaComputer Science ; Engineering
WOS SubjectComputer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS IDWOS:000384644000012
Citation statistics
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/12301
Collection复杂系统管理与控制国家重点实验室_平行控制
Corresponding AuthorLuo,Biao
Affiliation1.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China
2.Univ Sci & Technol Beijing, Sch Automat & Elect Engn, Beijing 100083, Peoples R China
3.Texas A&M Univ Qatar, Doha 23874, Qatar
Recommended Citation
GB/T 7714
Luo, Biao,Liu, Derong,Huang, Tingwen,et al. Model-Free Optimal Tracking Control via Critic-Only Q-Learning[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2016,27(10):2134-2144.
APA Luo, Biao,Liu, Derong,Huang, Tingwen,Wang, Ding,&Luo,Biao.(2016).Model-Free Optimal Tracking Control via Critic-Only Q-Learning.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,27(10),2134-2144.
MLA Luo, Biao,et al."Model-Free Optimal Tracking Control via Critic-Only Q-Learning".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 27.10(2016):2134-2144.
Files in This Item: Download All
File Name/Size DocType Version Access License
2016IEEE TNNLS Model(1521KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Luo, Biao]'s Articles
[Liu, Derong]'s Articles
[Huang, Tingwen]'s Articles
Baidu academic
Similar articles in Baidu academic
[Luo, Biao]'s Articles
[Liu, Derong]'s Articles
[Huang, Tingwen]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Luo, Biao]'s Articles
[Liu, Derong]'s Articles
[Huang, Tingwen]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: 2016IEEE TNNLS Model-Free Optimal Tracking Control via Critic-Only Q-Learning.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.