|MEC-A Near-Optimal Online Reinforcement Learning Algorithm for Continuous Deterministic Systems|
|Zhao, Dongbin; Zhu, Yuanheng
|发表期刊||IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
|摘要||In this paper, the first probably approximately correct (PAC) algorithm for continuous deterministic systems without relying on any system dynamics is proposed. It combines the state aggregation technique and the efficient exploration principle, and makes high utilization of online observed samples. We use a grid to partition the continuous state space into different cells to save samples. A near-upper Q operator is defined to produce a near-upper Q function using samples in each cell. The corresponding greedy policy effectively balances between exploration and exploitation. With the rigorous analysis, we prove that there is a polynomial time bound of executing nonoptimal actions in our algorithm. After finite steps, the final policy reaches near optimal in the framework of PAC. The implementation requires no knowledge of systems and has less computation complexity. Simulation studies confirm that it is a better performance than other similar PAC algorithms.|
Probably Approximately Correct (Pac)
Reinforcement Learning (Rl)
|WOS标题词||Science & Technology
; MODEL-BASED EXPLORATION
; ZERO-SUM GAMES
; CONTROL SCHEME
; UNKNOWN DYNAMICS
|WOS类目||Computer Science, Artificial Intelligence
; Computer Science, Hardware & Architecture
; Computer Science, Theory & Methods
; Engineering, Electrical & Electronic
|作者单位||Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China|
Zhao, Dongbin,Zhu, Yuanheng. MEC-A Near-Optimal Online Reinforcement Learning Algorithm for Continuous Deterministic Systems[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2015,26(2):346-356.
Zhao, Dongbin,&Zhu, Yuanheng.(2015).MEC-A Near-Optimal Online Reinforcement Learning Algorithm for Continuous Deterministic Systems.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,26(2),346-356.
Zhao, Dongbin,et al."MEC-A Near-Optimal Online Reinforcement Learning Algorithm for Continuous Deterministic Systems".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 26.2(2015):346-356.