Knowledge Commons of Institute of Automation,CAS
Subspace-Aware Exploration for Sparse-Reward Multi-Agent Tasks | |
Pei Xu1,2; Junge Zhang2; Qiyue Yin2; Chao Yu4; Yaodong Yang5,6; Kaiqi Huang1,2,3 | |
2023-02-14 | |
会议名称 | The 37th AAAI Conference on Artificial Intelligence |
会议日期 | 2023-2-7 |
会议地点 | Washington DC, USA |
出版者 | Association for the Advancement of Artificial Intelligence |
摘要 | Exploration under sparse rewards is a key challenge for multi agent reinforcement learning problems. One possible solution to this issue is to exploit inherent task structures for an acceleration of exploration. In this paper, we present a novel exploration approach, which encodes a special structural prior on the reward function into exploration, for sparse-reward multi agent tasks. Specifically, a novel entropic exploration objective which encodes the structural prior is proposed to accelerate the discovery of rewards. By maximizing the lower bound of this objective, we then propose an algorithm with moderate computational cost, which can be applied to practical tasks. Under the sparse-reward setting, we show that the proposed algorithm significantly outperforms the state-of-the-art algorithms in the multiple-particle environment, the Google Research Football and StarCraft II micromanagement tasks. To the best of our knowledge, on some hard tasks (such as 27m vs 30m) which have relatively larger number of agents and need non-trivial strategies to defeat enemies, our method is the first to learn winning strategies under the sparse-reward setting. |
关键词 | deep reinforcement learning sparse reward exploration multi-agent |
收录类别 | EI |
语种 | 英语 |
是否为代表性论文 | 否 |
七大方向——子方向分类 | 强化与进化学习 |
国重实验室规划方向分类 | 智能博弈与对手建模 |
是否有论文关联数据集需要存交 | 否 |
文献类型 | 会议论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/52050 |
专题 | 复杂系统认知与决策实验室_智能系统与工程 |
作者单位 | 1.School of Artificial Intelligence, University of Chinese Academy of Sciences 2.CRISE, Institute of Automation, Chinese Academy of Sciences 3.CAS, Center for Excellence in Brain Science and Intelligence Technology 4.School of Computer Science and Engineering, Sun Yat-sen University 5.Beijing Institute for General AI 6.Institute for AI, Peking University |
第一作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Pei Xu,Junge Zhang,Qiyue Yin,et al. Subspace-Aware Exploration for Sparse-Reward Multi-Agent Tasks[C]:Association for the Advancement of Artificial Intelligence,2023. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
aaai2023_copyright_s(2037KB) | 会议论文 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论