中国科学院自动化研究所机构知识库

Agent-agnostic reinforcement learning aims to learn
a universal control policy that can simultaneously control a set of
robots with different morphologies. Recent studies have suggested
that using the transformer model can address variations in
state and action spaces caused by different morphologies, and
morphology information is necessary to improve policy performance.
However, existing methods have limitations in exploiting
morphological information, where the rationality of observation
integration cannot be guaranteed. We propose Morphological
Adaptive Transformer (MAT), a transformer-based universal
control algorithm that can adapt to various morphologies without
any modifications. MAT includes two essential components:
functional position encoding and morphological attention mechanism.
The functional position encoding provides robust and
consistent positional prior information for limb observation to
avoid limb confusion and implicitly obtain functional descriptions
of limbs. The morphological attention mechanism enhances the
attribute prior information of limbs, improves the correlation
between observations and makes the policy pay attention to more
limbs. We combine observation with prior information to help
policy adapt to the morphology of robots, thereby optimizing its
performance with unknown morphologies. Experiments on agentagnostic
tasks in Gym MuJoCo environment demonstrate that
our algorithm can assign more reasonable morphological prior
information to each limb, and the performance of our algorithm
is comparable to the prior state-of-the-art algorithm with better
generalization.