An Empirical Study on Google Research Football Multi-agent Scenarios

doi:10.1007/s11633-023-1426-8

CASIA OpenIR > 学术期刊 > Machine Intelligence Research

	An Empirical Study on Google Research Football Multi-agent Scenarios
	Yan Song 1; He Jiang2 ; Zheng Tian 3; Haifeng Zhang 1; Yingping Zhang 4; Jiangcheng Zhu 4; Zonghong Dai 4; Weinan Zhang 5; Jun Wang 6
发表期刊	Machine Intelligence Research
ISSN	2731-538X
	2024
卷号	21 期号:3 页码:549-570
摘要	Few multi-agent reinforcement learning (MARL) researches on Google research football (GRF)[1] focus on the 11-vs-11 multi-agent full-game scenario and to the best of our knowledge, no open benchmark on this scenario has been released to the public. In this work, we fill the gap by providing a population-based MARL training pipeline and hyperparameter settings on multi-agent football scenario that outperforms the bot with difficulty 1.0 from scratch within 2 million steps. Our experiments serve as a reference for the expected performance of independent proximal policy optimization (IPPO)[2], a state-of-the-art multi-agent reinforcement learning algorithm where each agent tries to maximize its own policy independently across various training configurations. Meanwhile, we release our training framework Light-MALib which extends the MALib[3] codebase by distributed and asynchronous implementation with additional analytical tools for football games. Finally, we provide guidance for building strong football AI with population-based training[4] and release diverse pretrained policies for benchmarking. The goal is to provide the community with a head start for whoever experiment their works on GRF and a simple-to-use population-based training framework for further improving their agents through self-play. The implementation is available at https://github.com/Shanghai-Digital-Brain-Laboratory/DB-Football.
关键词	Multi-agent reinforcement learning (RL), distributed RL system, population-based training, reward shaping, game theory
DOI	10.1007/s11633-023-1426-8
引用统计
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/56482
专题	学术期刊_Machine Intelligence Research
作者单位	1.Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China 2.Digital Brain Lab, Shanghai 200001, China 3.ShanghaiTech University, Shanghai 200001, China 4.Huawei Cloud, Guiyang 550003, China 5.Shanghai Jiao Tong University, Shanghai 200001, China 6.University College London, London WC1E 6PT, UK
第一作者单位	中国科学院自动化研究所
推荐引用方式 GB/T 7714	Yan Song,He Jiang,Zheng Tian,et al. An Empirical Study on Google Research Football Multi-agent Scenarios[J]. Machine Intelligence Research,2024,21(3):549-570.
APA	Yan Song.,He Jiang.,Zheng Tian.,Haifeng Zhang.,Yingping Zhang.,...&Jun Wang.(2024).An Empirical Study on Google Research Football Multi-agent Scenarios.Machine Intelligence Research,21(3),549-570.
MLA	Yan Song,et al."An Empirical Study on Google Research Football Multi-agent Scenarios".Machine Intelligence Research 21.3(2024):549-570.

条目包含的文件
文件名称/大小	文献类型	版本类型	开放类型	使用许可
MIR-2022-11-356.pdf（24588KB）	期刊论文	出版稿	开放获取	CC BY-NC-SA	浏览