|MEC-A Near-Optimal Online Reinforcement Learning Algorithm for Continuous Deterministic Systems|
|Zhao, Dongbin; Zhu, Yuanheng
|Source Publication||IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
|Abstract||In this paper, the first probably approximately correct (PAC) algorithm for continuous deterministic systems without relying on any system dynamics is proposed. It combines the state aggregation technique and the efficient exploration principle, and makes high utilization of online observed samples. We use a grid to partition the continuous state space into different cells to save samples. A near-upper Q operator is defined to produce a near-upper Q function using samples in each cell. The corresponding greedy policy effectively balances between exploration and exploitation. With the rigorous analysis, we prove that there is a polynomial time bound of executing nonoptimal actions in our algorithm. After finite steps, the final policy reaches near optimal in the framework of PAC. The implementation requires no knowledge of systems and has less computation complexity. Simulation studies confirm that it is a better performance than other similar PAC algorithms.|
Probably Approximately Correct (Pac)
Reinforcement Learning (Rl)
|WOS Headings||Science & Technology
|WOS Keyword||TIME NONLINEAR-SYSTEMS
; MODEL-BASED EXPLORATION
; ZERO-SUM GAMES
; CONTROL SCHEME
; UNKNOWN DYNAMICS
|WOS Research Area||Computer Science
|WOS Subject||Computer Science, Artificial Intelligence
; Computer Science, Hardware & Architecture
; Computer Science, Theory & Methods
; Engineering, Electrical & Electronic
|Affiliation||Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China|
|First Author Affilication||Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China
Zhao, Dongbin,Zhu, Yuanheng. MEC-A Near-Optimal Online Reinforcement Learning Algorithm for Continuous Deterministic Systems[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2015,26(2):346-356.
Zhao, Dongbin,&Zhu, Yuanheng.(2015).MEC-A Near-Optimal Online Reinforcement Learning Algorithm for Continuous Deterministic Systems.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,26(2),346-356.
Zhao, Dongbin,et al."MEC-A Near-Optimal Online Reinforcement Learning Algorithm for Continuous Deterministic Systems".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 26.2(2015):346-356.
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.