中国科学院自动化研究所机构知识库

Many real-world applications involve some agents
that fall into two teams, with payoffs that are equal within the
same team but of opposite sign across the opponent team. The
so-called two-team zero-sum Markov games (2t0sMGs) can be
resolved with reinforcement learning in recent years. However,
existing methods are thus inefficient in light of insufficient consideration
of intra-team credit assignment, data utilization, and computational
intractability. In this paper, we propose the individualglobal-
minimax(IGMM)principle to ensure the coherence between
two-team minimax behaviors and the individual greedy behaviors
through Q functions in 2t0sMGs. Based on it, we present a novel
multi-agent reinforcement learning framework, Factorized Multi-
AgentMiniMax Q-Learning (FM3Q), which can factorize the joint
minimax Q function into individual ones and iteratively solve for
the IGMM-satisfied minimax Q functions for 2t0sMGs. Moreover,
an online learning algorithm with neural networks is proposed to
implement FM3Q and obtain the deterministic and decentralized
minimax policies for two-team players. A theoretical analysis is
provided to prove the convergence of FM3Q. Empirically, we use
three environments to evaluate the learning efficiency and final
performance of FM3Q and show its superiority on 2t0sMGs.