Knowledge Commons of Institute of Automation,CAS
Keep Various Trajectories: Promoting Exploration of Ensemble Policies in Continuous Control | |
Chao Li1,4; Chen Gong2; Qiang He3; Xinwen Hou1,4 | |
2023-09 | |
会议名称 | the Thirty-seventh Annual Conference on Neural Information Processing Systems |
会议录名称 | Advances in Neural Information Processing Systems |
卷号 | 36 |
页码 | 5223--5235 |
会议日期 | 2023-12-10 |
会议地点 | New Orleans, USA |
会议录编者/会议主办者 | A. Oh and T. Naumann and A. Globerson and K. Saenko and M. Hardt and S. Levine |
出版者 | Curran Associates, Inc. |
摘要 | The combination of deep reinforcement learning (DRL) with ensemble methods has been proved to be highly effective in addressing complex sequential decision- making problems. This success can be primarily attributed to the utilization of multiple models, which enhances both the robustness of the policy and the accuracy of value function estimation. However, there has been limited analysis of the empirical success of current ensemble RL methods thus far. Our new analysis reveals that the sample efficiency of previous ensemble DRL algorithms may be limited by sub-policies that are not as diverse as they could be. Motivated by these findings, our study introduces a new ensemble RL algorithm, termed Trajectories-awarE Ensemble exploratioN (TEEN). The primary goal of TEEN is to maximize the expected return while promoting more diverse trajectories. Through extensive experiments, we demonstrate that TEEN not only enhances the sample diversity of the ensemble policy compared to using sub-policies alone but also improves the performance over ensemble RL algorithms. On average, TEEN outperforms the baseline ensemble DRL algorithms by 41% in performance on the tested representative environments. |
URL | 查看原文 |
收录类别 | 其他 |
语种 | 英语 |
是否为代表性论文 | 是 |
七大方向——子方向分类 | 强化与进化学习 |
国重实验室规划方向分类 | 认知决策知识体系 |
是否有论文关联数据集需要存交 | 否 |
文献类型 | 会议论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/56695 |
专题 | 多模态人工智能系统全国重点实验室_机器人理论与应用 |
通讯作者 | Xinwen Hou |
作者单位 | 1.Institute of Automation, Chinese Academy of Sciences, China 2.University of Virginia, USA 3.Ruhr University Bochum, Germany 4.University of Chinese Academy of Sciences, China |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Chao Li,Chen Gong,Qiang He,et al. Keep Various Trajectories: Promoting Exploration of Ensemble Policies in Continuous Control[C]//A. Oh and T. Naumann and A. Globerson and K. Saenko and M. Hardt and S. Levine:Curran Associates, Inc.,2023:5223--5235. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
NeurIPS-2023-keep-va(1457KB) | 会议论文 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论