Knowledge Commons of Institute of Automation,CAS
An Empirical Study on Google Research Football Multi-agent Scenarios | |
Yan Song1; He Jiang2; Zheng Tian3; Haifeng Zhang1; Yingping Zhang4; Jiangcheng Zhu4; Zonghong Dai4; Weinan Zhang5; Jun Wang6 | |
发表期刊 | Machine Intelligence Research |
ISSN | 2731-538X |
2024 | |
卷号 | 21期号:3页码:549-570 |
摘要 | Few multi-agent reinforcement learning (MARL) researches on Google research football (GRF)[1] focus on the 11-vs-11 multi-agent full-game scenario and to the best of our knowledge, no open benchmark on this scenario has been released to the public. In this work, we fill the gap by providing a population-based MARL training pipeline and hyperparameter settings on multi-agent football scenario that outperforms the bot with difficulty 1.0 from scratch within 2 million steps. Our experiments serve as a reference for the expected performance of independent proximal policy optimization (IPPO)[2], a state-of-the-art multi-agent reinforcement learning algorithm where each agent tries to maximize its own policy independently across various training configurations. Meanwhile, we release our training framework Light-MALib which extends the MALib[3] codebase by distributed and asynchronous implementation with additional analytical tools for football games. Finally, we provide guidance for building strong football AI with population-based training[4] and release diverse pretrained policies for benchmarking. The goal is to provide the community with a head start for whoever experiment their works on GRF and a simple-to-use population-based training framework for further improving their agents through self-play. The implementation is available at https://github.com/Shanghai-Digital-Brain-Laboratory/DB-Football. |
关键词 | Multi-agent reinforcement learning (RL), distributed RL system, population-based training, reward shaping, game theory |
DOI | 10.1007/s11633-023-1426-8 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/56482 |
专题 | 学术期刊_Machine Intelligence Research |
作者单位 | 1.Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China 2.Digital Brain Lab, Shanghai 200001, China 3.ShanghaiTech University, Shanghai 200001, China 4.Huawei Cloud, Guiyang 550003, China 5.Shanghai Jiao Tong University, Shanghai 200001, China 6.University College London, London WC1E 6PT, UK |
第一作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Yan Song,He Jiang,Zheng Tian,et al. An Empirical Study on Google Research Football Multi-agent Scenarios[J]. Machine Intelligence Research,2024,21(3):549-570. |
APA | Yan Song.,He Jiang.,Zheng Tian.,Haifeng Zhang.,Yingping Zhang.,...&Jun Wang.(2024).An Empirical Study on Google Research Football Multi-agent Scenarios.Machine Intelligence Research,21(3),549-570. |
MLA | Yan Song,et al."An Empirical Study on Google Research Football Multi-agent Scenarios".Machine Intelligence Research 21.3(2024):549-570. |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
MIR-2022-11-356.pdf(24588KB) | 期刊论文 | 出版稿 | 开放获取 | CC BY-NC-SA | 浏览 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论