CASIA OpenIR  > 复杂系统认知与决策实验室  > 听觉模型与认知计算
Learning in bi-level markov games
Meng Linghui1,2; Ruan Jingqing1,2; Xing Dengpeng1; Xu Bo1,2
2022-07
Conference Name2022 International Joint Conference on Neural Networks (IJCNN).
Conference Date2022.7.18-2022.7.23
Conference PlacePadua, Italy
Abstract

Although multi-agent reinforcement learning (MARL) has demonstrated remarkable progress in tackling sophisticated cooperative tasks, the assumption that agents take simultaneous actions still limits the applicability of MARL for many real-world problems. In this work, we relax the assumption by proposing the framework of the bi-level Markov game (BMG). BMG breaks the simultaneity by assigning two players with a leader-follower relationship in which the leader considers the policy of the follower who is taking the best response based on the leader's actions. We propose two provably convergent algorithms to solve BMG: BMG-1 and BMG-2. The former uses the standard Q-learning, while the latter relieves solving the local Stackelberg equilibrium in BMG-1 with the further two-step transition to estimate the state value. For both methods, we consider temporal difference learning techniques with both tabular and neural network representations. To verify the effectiveness of our BMG framework, we test on a series of games, including Seeker, Cooperative Navigation, and Football, that are challenging to existing MARL solvers find challenging to solve: Seeker, Cooperative Navigation, and Football. Experimental results show that our BMG methods achieve competitive advantages in terms of better performance and lower variance.  

Sub direction classification多智能体系统
planning direction of the national heavy laboratory多智能体决策
Paper associated data
Document Type会议论文
Identifierhttp://ir.ia.ac.cn/handle/173211/57335
Collection复杂系统认知与决策实验室_听觉模型与认知计算
Affiliation1.Institute of Automation, Chinese Academy of Sciences
2.School of Artificial Intelligence, University of Chinese Academy of Sciences
First Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Meng Linghui,Ruan Jingqing,Xing Dengpeng,et al. Learning in bi-level markov games[C],2022.
Files in This Item: Download All
File Name/Size DocType Version Access License
bmg_full_paper.pdf(1450KB)会议论文 开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Meng Linghui]'s Articles
[Ruan Jingqing]'s Articles
[Xing Dengpeng]'s Articles
Baidu academic
Similar articles in Baidu academic
[Meng Linghui]'s Articles
[Ruan Jingqing]'s Articles
[Xing Dengpeng]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Meng Linghui]'s Articles
[Ruan Jingqing]'s Articles
[Xing Dengpeng]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: bmg_full_paper.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.