Efficient Hierarchical Reinforcement Learning via Mutual Information Constrained Subgoal Discovery
Kaishen Wang1,2; Jingqing Ruan2; Qingyang Zhang2; Dengpeng Xing1,2
2023-11
Conference NameInternational Conference on Neural Information Processing
Conference Date2023-11
Conference Place长沙
Abstract

Goal-conditioned hierarchical reinforcement learning has demonstrated impressive capabilities in addressing complex and long-horizon tasks. However, the extensive subgoal space often results in low sample efficiency and challenging exploration. To address this issue, we extract informative subgoals by constraining their generation range in mutual information distance space. Specifically, we impose two constraints on the high-level policy during off-policy training: the generated subgoals should be reached with less effort by the low-level policy, and the realization of these subgoals can facilitate achieving the desired goals. These two constraints enable subgoals to act as critical links between the current states and the desired goals, providing more effective guidance to the low-level policy. The empirical results on continuous control tasks
demonstrate that our proposed method significantly enhances the training efficiency, regardless of the dimensions of the state and action spaces, while ensuring comparable performance to state-of-the-art methods.

Indexed ByEI
Language英语
Sub direction classification强化与进化学习
planning direction of the national heavy laboratory认知决策知识体系
Paper associated data
Document Type会议论文
Identifierhttp://ir.ia.ac.cn/handle/173211/56577
Collection多模态人工智能系统全国重点实验室_智能机器人系统研究
Corresponding AuthorDengpeng Xing
Affiliation1.University of Chinese Academy of Sciences
2.Institute of Automation, Chinese Academy of Sciences
First Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Corresponding Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Kaishen Wang,Jingqing Ruan,Qingyang Zhang,et al. Efficient Hierarchical Reinforcement Learning via Mutual Information Constrained Subgoal Discovery[C],2023.
Files in This Item: Download All
File Name/Size DocType Version Access License
已发表1.pdf(2044KB)会议论文 开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Kaishen Wang]'s Articles
[Jingqing Ruan]'s Articles
[Qingyang Zhang]'s Articles
Baidu academic
Similar articles in Baidu academic
[Kaishen Wang]'s Articles
[Jingqing Ruan]'s Articles
[Qingyang Zhang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Kaishen Wang]'s Articles
[Jingqing Ruan]'s Articles
[Qingyang Zhang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: 已发表1.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.