Knowledge Commons of Institute of Automation,CAS
Prototypical context-aware dynamics generalization for high-dimensional model-based reinforcement learning | |
Junjie, Wang1,2; Yao, Mu3; Dong, Li4; Qichao,Zhang1,2; Dongbin, Zhao1,2; Yuzheng, Zhuang4; Ping, Luo3; Bin, Wang4; Jianye, Hao4 | |
2023-05 | |
会议名称 | International Conference on Learning Representations, Workshop on Scene Representations for Autonomous Driving |
会议日期 | 2023-5-5 |
会议地点 | Kigali City, Rwanda, Africa |
摘要 | The ability to generalize different dynamics is crucial for decision-making in autonomous driving that relies on high-dimensional inputs. The latent world model provides a promising way to learn policies in a compact latent space for tasks with high-dimensional observations, however, its generalization across diverse environments with unseen dynamics remains challenging. Although the recurrent structure utilized in current advances helps to capture local dynamics, modeling only state transitions without an explicit understanding of environmental context limits the generalization ability of the dynamics model. To address this issue, we propose a Prototypical Context-Aware Dynamics (ProtoCAD) model, which captures the local dynamics by temporally consistent latent context and enables dynamics generalization in high-dimensional control tasks. ProtoCAD extracts useful contextual information with the prototypes clustered over the batch, and it benefits model-based reinforcement learning in two ways: 1) A temporally consistent prototypes regularizer is utilized, which encourages the prototype assignments produced for different temporal parts of the same latent trajectory to be temporally consistent instead of comparing the features; 2) A context representation is designed, which combines both projection embedding of latent states and aggregated prototypes and can significantly improve the dynamics generalization ability. Extensive experiments show that ProtoCAD surpasses existing methods in terms of dynamics generalization. |
七大方向——子方向分类 | 强化与进化学习 |
国重实验室规划方向分类 | 智能计算与学习 |
是否有论文关联数据集需要存交 | 否 |
文献类型 | 会议论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/52274 |
专题 | 多模态人工智能系统全国重点实验室_深度强化学习 |
作者单位 | 1.State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences 2.University of Chinese Academy of Sciences 3.The University of Hong Kong 4.Huawei Noah’s Ark Lab |
第一作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Junjie, Wang,Yao, Mu,Dong, Li,et al. Prototypical context-aware dynamics generalization for high-dimensional model-based reinforcement learning[C],2023. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
ICLR_2023_SR4AD.pdf(3492KB) | 会议论文 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论