Efficient Exploration for Multi-Agent Reinforcement Learning via Transferable Successor Features

doi:10.1109/JAS.2022.105809

CASIA OpenIR > 学术期刊 > IEEE/CAA Journal of Automatica Sinica

	Efficient Exploration for Multi-Agent Reinforcement Learning via Transferable Successor Features
	Wenzhang Liu; Lu Dong; Dan Niu; Changyin Sun
发表期刊	IEEE/CAA Journal of Automatica Sinica
ISSN	2329-9266
	2022
卷号	9 期号:9 页码:1673-1686
摘要	In multi-agent reinforcement learning (MARL), the behaviors of each agent can influence the learning of others, and the agents have to search in an exponentially enlarged joint-action space. Hence, it is challenging for the multi-agent teams to explore in the environment. Agents may achieve suboptimal policies and fail to solve some complex tasks. To improve the exploring efficiency as well as the performance of MARL tasks, in this paper, we propose a new approach by transferring the knowledge across tasks. Differently from the traditional MARL algorithms, we first assume that the reward functions can be computed by linear combinations of a shared feature function and a set of task-specific weights. Then, we define a set of basic MARL tasks in the source domain and pre-train them as the basic knowledge for further use. Finally, once the weights for target tasks are available, it will be easier to get a well-performed policy to explore in the target domain. Hence, the learning process of agents for target tasks is speeded up by taking full use of the basic knowledge that was learned previously. We evaluate the proposed algorithm on two challenging MARL tasks: cooperative box-pushing and non-monotonic predator-prey. The experiment results have demonstrated the improved performance compared with state-of-the-art MARL algorithms.
关键词	Knowledge transfer multi-agent systems reinforcement learning successor features
DOI	10.1109/JAS.2022.105809
引用统计	被引频次：8[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/49681
专题	学术期刊_IEEE/CAA Journal of Automatica Sinica
推荐引用方式 GB/T 7714	Wenzhang Liu,Lu Dong,Dan Niu,et al. Efficient Exploration for Multi-Agent Reinforcement Learning via Transferable Successor Features[J]. IEEE/CAA Journal of Automatica Sinica,2022,9(9):1673-1686.
APA	Wenzhang Liu,Lu Dong,Dan Niu,&Changyin Sun.(2022).Efficient Exploration for Multi-Agent Reinforcement Learning via Transferable Successor Features.IEEE/CAA Journal of Automatica Sinica,9(9),1673-1686.
MLA	Wenzhang Liu,et al."Efficient Exploration for Multi-Agent Reinforcement Learning via Transferable Successor Features".IEEE/CAA Journal of Automatica Sinica 9.9(2022):1673-1686.

条目包含的文件		下载所有文件
文件名称/大小	文献类型	版本类型	开放类型	使用许可
JAS-2021-1147.pdf（5554KB）	期刊论文	出版稿	开放获取	CC BY-NC-SA	浏览下载