Knowledge Commons of Institute of Automation,CAS
Pseudo Value Network Distillation for High-Performance Exploration | |
Zhao EM(赵恩民)1,2![]() ![]() ![]() ![]() | |
2023-04 | |
会议名称 | International Joint Conference on Neural Networks |
会议日期 | 2023-06 |
会议地点 | 澳大利亚 |
摘要 | Solving hard exploration tasks with sparse rewards is notoriously challenging in reinforcement learning (RL), which needs to address two key issues simultaneously: exploiting past successful experiences and exploring the unknown environment. Many prior works take expert demonstrations as successful experiences and learn to imitate them directly. However, these demonstrations are often not available in practice. Recently, curiosity-driven RL methods provide intrinsic rewards, encouraging the agent to explore states with high novelty. Nonetheless, they lack a mechanism for leveraging past good experiences effectively. This work presents a Pseudo Value Network Distillation (PVND) framework to balance the RL agent's exploitative and exploratory behaviors effectively and automatically. In particular, PVND learns to set high exploitation bonuses to the critical states in rewarded trajectories from past experiences and high exploration bonuses to the novel states that agents rarely visit during exploration. We theoretically demonstrate that PVND gives larger positive intrinsic rewards to more critical states. Furthermore, PVND automatically finds meaningful and critical hierarchical sub-tasks for agents to accomplish the final goal progressively. Competitive results in several hard exploration sparse reward problems have verified its effectiveness and efficiency. |
学科门类 | 工学 |
DOI | 无 |
URL | 查看原文 |
收录类别 | EI |
语种 | 英语 |
是否为代表性论文 | 否 |
七大方向——子方向分类 | 机器学习 |
国重实验室规划方向分类 | 开放博弈基础理论 |
是否有论文关联数据集需要存交 | 否 |
引用统计 | |
文献类型 | 会议论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/52243 |
专题 | 复杂系统认知与决策实验室_决策指挥与体系智能 |
通讯作者 | Xing JL(兴军亮) |
作者单位 | 1.Institute of Automation, Chinese Academy of Sciences 2.School of Artificial Intelligence, University of Chinese Academy of Sciences 3.Department of Computer Science and Technology, Tsinghua University |
第一作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Zhao EM,Xing JL,Li K,et al. Pseudo Value Network Distillation for High-Performance Exploration[C],2023. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
IJCNN23_PVND.pdf(5874KB) | 会议论文 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论