Knowledge Commons of Institute of Automation,CAS
Attention Enhanced Reinforcement Learning for Multi agent Cooperation | |
Pu, Zhiqiang1; Wang, Huimu1,2; Liu, Zhen1; Yi, Jianqiang1; Wu, Shiguang1 | |
发表期刊 | IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS |
ISSN | 2162-237X |
2022-02-17 | |
页码 | 15 |
摘要 | In this article, a novel method, called attention enhanced reinforcement learning (AERL), is proposed to address issues including complex interaction, limited communication range, and time-varying communication topology for multi agent cooperation. AERL includes a communication enhanced network (CEN), a graph spatiotemporal long short-term memory network (GST-LSTM), and parameters sharing multi-pseudo critic proximal policy optimization (PS-MPC-PPO). Specifically, CEN based on graph attention mechanism is designed to enlarge the agents' communication range and to deal with complex interaction among the agents. GST-LSTM, which replaces the standard fully connected (FC) operator in LSTM with graph attention operator, is designed to capture the temporal dependence while maintaining the spatial structure learned by CEN. PS-MPC-PPO, which extends proximal policy optimization (PPO) in multi agent systems with parameters' sharing to scale to environments with a large number of agents in training, is designed with multi-pseudo critics to mitigate the bias problem in training and accelerate the convergence process. Simulation results for three groups of representative scenarios including formation control, group containment, and predator-prey games demonstrate the effectiveness and robustness of AERL. |
关键词 | Training Reinforcement learning Games Scalability Task analysis Standards Optimization Attention mechanism deep reinforcement learning (DRL) graph convolutional networks multi agent systems |
DOI | 10.1109/TNNLS.2022.3146858 |
关键词[WOS] | LEVEL ; GAME ; GO |
收录类别 | SCI |
语种 | 英语 |
资助项目 | National Key Research and Development Program of China[2018AAA0102404] ; National Natural Science Foundation of China[62073323] ; National Natural Science Foundation of China[61806199] ; Strategic Priority Research Program of Chinese Academy of Sciences[XDA27030403] ; External Cooperation Key Project of Chinese Academy Sciences[173211KYSB20200002] |
项目资助者 | National Key Research and Development Program of China ; National Natural Science Foundation of China ; Strategic Priority Research Program of Chinese Academy of Sciences ; External Cooperation Key Project of Chinese Academy Sciences |
WOS研究方向 | Computer Science ; Engineering |
WOS类目 | Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic |
WOS记录号 | WOS:000761254200001 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
七大方向——子方向分类 | 强化与进化学习 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/47922 |
专题 | 复杂系统认知与决策实验室_飞行器智能技术 |
通讯作者 | Wang, Huimu |
作者单位 | 1.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Pu, Zhiqiang,Wang, Huimu,Liu, Zhen,et al. Attention Enhanced Reinforcement Learning for Multi agent Cooperation[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2022:15. |
APA | Pu, Zhiqiang,Wang, Huimu,Liu, Zhen,Yi, Jianqiang,&Wu, Shiguang.(2022).Attention Enhanced Reinforcement Learning for Multi agent Cooperation.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,15. |
MLA | Pu, Zhiqiang,et al."Attention Enhanced Reinforcement Learning for Multi agent Cooperation".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2022):15. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
Attention_Enhanced_R(2967KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论