Knowledge Commons of Institute of Automation,CAS
Sparse online kernelized actor-critic Learning in reproducing kernel Hilbert space | |
Yang, Yongliang1; Zhu, Hufei1; Zhang, Qichao2,3; Zhao, Bo4; Li, Zhenning5; Wunsch, Donald C.6 | |
发表期刊 | ARTIFICIAL INTELLIGENCE REVIEW |
ISSN | 0269-2821 |
2021-08-07 | |
页码 | 36 |
通讯作者 | Zhao, Bo(zhaobo@bnu.edu.cn) |
摘要 | In this paper, we develop a novel non-parametric online actor-critic reinforcement learning (RL) algorithm to solve optimal regulation problems for a class of continuous-time affine nonlinear dynamical systems. To deal with the value function approximation (VFA) with inherent nonlinear and unknown structure, a reproducing kernel Hilbert space (RKHS)-based kernelized method is designed through online sparsification, where the dictionary size is fixed and consists of updated elements. In addition, the linear independence check condition, i.e., an online criteria, is designed to determine whether the online data should be inserted into the dictionary. The RHKS-based kernelized VFA has a variable structure in accordance with the online data collection, which is different from classical parametric VFA methods with a fixed structure. Furthermore, we develop a sparse online kernelized actor-critic learning RL method to learn the unknown optimal value function and the optimal control policy in an adaptive fashion. The convergence of the presented kernelized actor-critic learning method to the optimum is provided. The boundedness of the closed-loop signals during the online learning phase can be guaranteed. Finally, a simulation example is conducted to demonstrate the effectiveness of the presented kernelized actor-critic learning algorithm. |
关键词 | Reproducing kernel Hilbert space Actor-critic learning Value function approximation Online sparsification Non-parametric learning |
DOI | 10.1007/s10462-021-10045-9 |
关键词[WOS] | FAULT-TOLERANT CONTROL ; NONLINEAR-SYSTEMS ; TRACKING CONTROL ; APPROXIMATION |
收录类别 | SCI |
语种 | 英语 |
资助项目 | National Natural Science Foundation of China[61903028] ; National Natural Science Foundation of China[61973330] ; National Natural Science Foundation of China[61803371] ; National Natural Science Foundation of China[61773075] ; Beijing Natural Science Foundation[4212038] ; Open Research Project of the State Key Laboratory of Management and Control for Complex Systems, Institute of Sciences[20210108] ; Open Research Project of the State Key Laboratory of Industrial Control Technology, Zhejiang University, China[ICT2021B48] ; Fundamental Research Funds for the Central Universities[2019NTST25] ; State Key Laboratory of Synthetical Automation for Process Industries[2019-KF-23-03] |
项目资助者 | National Natural Science Foundation of China ; Beijing Natural Science Foundation ; Open Research Project of the State Key Laboratory of Management and Control for Complex Systems, Institute of Sciences ; Open Research Project of the State Key Laboratory of Industrial Control Technology, Zhejiang University, China ; Fundamental Research Funds for the Central Universities ; State Key Laboratory of Synthetical Automation for Process Industries |
WOS研究方向 | Computer Science |
WOS类目 | Computer Science, Artificial Intelligence |
WOS记录号 | WOS:000682662600001 |
出版者 | SPRINGER |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/45683 |
专题 | 多模态人工智能系统全国重点实验室_深度强化学习 |
通讯作者 | Zhao, Bo |
作者单位 | 1.Univ Sci & Technol Beijing, Sch Automat & Elect Engn, Beijing 100083, Peoples R China 2.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China 3.Univ Chinese Acad Sci, Beijing, Peoples R China 4.Beijing Normal Univ, Sch Syst Sci, Beijing 100875, Peoples R China 5.Univ Macau, State Key Lab Internet Things Smart City, Taipa 59193, Macao, Peoples R China 6.Missouri Univ Sci & Technol, Dept Elect & Comp Engn, Rolla, MO 65401 USA |
推荐引用方式 GB/T 7714 | Yang, Yongliang,Zhu, Hufei,Zhang, Qichao,et al. Sparse online kernelized actor-critic Learning in reproducing kernel Hilbert space[J]. ARTIFICIAL INTELLIGENCE REVIEW,2021:36. |
APA | Yang, Yongliang,Zhu, Hufei,Zhang, Qichao,Zhao, Bo,Li, Zhenning,&Wunsch, Donald C..(2021).Sparse online kernelized actor-critic Learning in reproducing kernel Hilbert space.ARTIFICIAL INTELLIGENCE REVIEW,36. |
MLA | Yang, Yongliang,et al."Sparse online kernelized actor-critic Learning in reproducing kernel Hilbert space".ARTIFICIAL INTELLIGENCE REVIEW (2021):36. |
条目包含的文件 | 条目无相关文件。 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论