Cognition-Driven Multiagent Policy Learning Framework for Promoting Cooperation

doi:10.1109/TG.2022.3186386

	Cognition-Driven Multiagent Policy Learning Framework for Promoting Cooperation
	Pu, Zhiqiang1,2 ; Wang, Huimu3 ; Liu, Boyin 1,2; Yi, Jianqiang1,2
发表期刊	IEEE TRANSACTIONS ON GAMES
ISSN	2475-1502
	2023-09-01
卷号	15 期号:3 页码:388-398
通讯作者	Pu, Zhiqiang(zhiqiang.pu@ia.ac.cn)
摘要	Many attempts have been made to promote cooperation for multiagent systems. However, several issues that draw less attentions but may dramatically degrade the cooperation performance still exist, such as redundant information interactions among neighbors, and difficulties in understanding complex and dynamic environments from high-level cognition. To address these limitations, a cognition-driven multiagent policy (CDMAP) learning framework is proposed in this article. It includes a cognition difference network (CDN), a coupling cognition network (CCN), and a policy optimization network (PON). CDN is designed based on a variational autoencoder, where a concept of cognition difference is defined to prune redundant interactions among agents for more efficient communication. Based on the pruned topology, CCN captures the hidden representations of the surrounding environment. Several coupling graph attention layers are incorporated in CCN, each layer with different but coupling adjacent matrices, yielding a comprehensive state understanding from multiple representation spaces. Based on the captured hidden states, PON generates the final policies, where QMIX is adopted as a value factorization method to alleviate the credit-assignment problem. At last, CDMAP is evaluated through two representative multiagent games including Google Research Football and StarCraft II. The results demonstrate its superior effectiveness compared with existing methods.
关键词	Cognition difference coupling cognition network (CCN) deep reinforcement learning (DRL) graph convolutional network multiagent systems (MASs)
DOI	10.1109/TG.2022.3186386
关键词[WOS]	GAME
收录类别	SCI
语种	英语
资助项目	National Key Research Development Program of China ; Strategic Priority Research Program of Chinese Academy of Sciences[2020AAA0103404] ; External Cooperation Key Project of Chinese Academy Sciences[XDA27030204] ; Science and Technology Development Fund of Macau[173211KYSB20200002] ; [0025/2019/AKP]
项目资助者	National Key Research Development Program of China ; Strategic Priority Research Program of Chinese Academy of Sciences ; External Cooperation Key Project of Chinese Academy Sciences ; Science and Technology Development Fund of Macau
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Software Engineering
WOS记录号	WOS:001068900800007
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/53060
专题	复杂系统认知与决策实验室
通讯作者	Pu, Zhiqiang
作者单位	1.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China 3.JD COM, Beijing 100176, Peoples R China
第一作者单位	中国科学院自动化研究所
通讯作者单位	中国科学院自动化研究所
推荐引用方式 GB/T 7714	Pu, Zhiqiang,Wang, Huimu,Liu, Boyin,et al. Cognition-Driven Multiagent Policy Learning Framework for Promoting Cooperation[J]. IEEE TRANSACTIONS ON GAMES,2023,15(3):388-398.
APA	Pu, Zhiqiang,Wang, Huimu,Liu, Boyin,&Yi, Jianqiang.(2023).Cognition-Driven Multiagent Policy Learning Framework for Promoting Cooperation.IEEE TRANSACTIONS ON GAMES,15(3),388-398.
MLA	Pu, Zhiqiang,et al."Cognition-Driven Multiagent Policy Learning Framework for Promoting Cooperation".IEEE TRANSACTIONS ON GAMES 15.3(2023):388-398.