Knowledge Commons of Institute of Automation,CAS
Real-Sim-Real Transfer for Real-World Robot Control Policy Learning with Deep Reinforcement Learning | |
Liu, Naijun1,2; Cai, Yinghao1; Lu, Tao1; Wang, Rui1,3; Wang, Shuo1,2,4 | |
发表期刊 | APPLIED SCIENCES-BASEL |
2020-03-01 | |
卷号 | 10期号:5页码:16 |
摘要 | Compared to traditional data-driven learning methods, recently developed deep reinforcement learning (DRL) approaches can be employed to train robot agents to obtain control policies with appealing performance. However, learning control policies for real-world robots through DRL is costly and cumbersome. A promising alternative is to train policies in simulated environments and transfer the learned policies to real-world scenarios. Unfortunately, due to the reality gap between simulated and real-world environments, the policies learned in simulated environments often cannot be generalized well to the real world. Bridging the reality gap is still a challenging problem. In this paper, we propose a novel real-sim-real (RSR) transfer method that includes a real-to-sim training phase and a sim-to-real inference phase. In the real-to-sim training phase, a task-relevant simulated environment is constructed based on semantic information of the real-world scenario and coordinate transformation, and then a policy is trained with the DRL method in the built simulated environment. In the sim-to-real inference phase, the learned policy is directly applied to control the robot in real-world scenarios without any real-world data. Experimental results in two different robot control tasks show that the proposed RSR method can train skill policies with high generalization performance and significantly low training costs. |
关键词 | robot policy learning reality gap simulated environment deep reinforcement learning |
DOI | 10.3390/app10051555 |
关键词[WOS] | DOMAIN ADAPTATION |
收录类别 | SCI |
语种 | 英语 |
资助项目 | National Natural Science Foundation of China[61773378] ; National Natural Science Foundation of China[U1713222] ; National Natural Science Foundation of China[U1806204] ; Equipment Pre-Research Field Fund[61403120407] ; Opening Project of Guangdong Provincial Key Lab of Robotics and Intelligent System |
项目资助者 | National Natural Science Foundation of China ; Equipment Pre-Research Field Fund ; Opening Project of Guangdong Provincial Key Lab of Robotics and Intelligent System |
WOS研究方向 | Chemistry ; Engineering ; Materials Science ; Physics |
WOS类目 | Chemistry, Multidisciplinary ; Engineering, Multidisciplinary ; Materials Science, Multidisciplinary ; Physics, Applied |
WOS记录号 | WOS:000525298100003 |
出版者 | MDPI |
七大方向——子方向分类 | 智能机器人 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/38869 |
专题 | 智能机器人系统研究 |
通讯作者 | Cai, Yinghao; Lu, Tao; Wang, Shuo |
作者单位 | 1.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100190, Peoples R China 3.Chinese Acad Sci, Shenzhen Inst Adv Technol, Guangdong Prov Key Lab Robot & Intelligent Syst, Shenzhen 518055, Peoples R China 4.Chinese Acad Sci, Ctr Excellence Brain Sci & Intelligence Technol, Shanghai 200031, Peoples R China |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Liu, Naijun,Cai, Yinghao,Lu, Tao,et al. Real-Sim-Real Transfer for Real-World Robot Control Policy Learning with Deep Reinforcement Learning[J]. APPLIED SCIENCES-BASEL,2020,10(5):16. |
APA | Liu, Naijun,Cai, Yinghao,Lu, Tao,Wang, Rui,&Wang, Shuo.(2020).Real-Sim-Real Transfer for Real-World Robot Control Policy Learning with Deep Reinforcement Learning.APPLIED SCIENCES-BASEL,10(5),16. |
MLA | Liu, Naijun,et al."Real-Sim-Real Transfer for Real-World Robot Control Policy Learning with Deep Reinforcement Learning".APPLIED SCIENCES-BASEL 10.5(2020):16. |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
85Real-Sim-Real Tran(6287KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论