Encoding Primitives Generation Policy Learning for Robotic Arm to Overcome Catastrophic Forgetting in Sequential Multi-tasks Learning
Fangzhou Xiong1,2; Zhiyong Liu1,2,3; Kaizhu Huang4,5; Xu Yang1,2; Hong Qiao1,2,3; Amir Hussain6
Source PublicationNeural Networks
2020-06
Issue2020.06.003Pages:12
Abstract
Continual learning, a widespread ability in people and animals, aims to learn and acquire new knowledge and skills continuously. Catastrophic forgetting usually occurs in continual learning when an agent attempts to learn different tasks sequentially without storing or accessing previous task information. Unfortunately, current learning systems, e.g., neural networks, are prone to deviate the weights learned in previous tasks after training new tasks, leading to catastrophic forgetting, especially in a sequential multi-tasks scenario. To address this problem, in this paper, we propose to overcome catastrophic forgetting with the focus on learning a series of robotic tasks sequentially. Particularly, a novel hierarchical neural network's framework called Encoding Primitives Generation Policy Learning (E-PGPL) is developed to enable continual learning with two components. By employing a variational autoencoder to project the original state space into a meaningful low-dimensional feature space, representative state primitives could be sampled to help learn corresponding policies for different tasks. In learning a new task, the feature space is required to be close to the previous ones so that previously learned tasks can be protected. Extensive experiments on several simulated robotic tasks demonstrate our method's efficacy to learn control policies for handling sequentially arriving multi-tasks, delivering improvement substantially over some other continual learning methods, especially for the tasks with more diversity.
KeywordSequential multi-tasks learning, Continual learning, Catastrophic forgetting, Robotics
Indexed BySCI
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/39080
Collection复杂系统管理与控制国家重点实验室_机器人理论与应用
Corresponding AuthorZhiyong Liu
Affiliation1.State Key Lab of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Science, 100190, Beijing, China
2.School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS), 100049, Beijing, China
3.CAS Centre for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, 200031, Shanghai, China
4.Department of EEE, Xi'an Jiaotong-Liverpool University, 215123, Suzhou, China
5.Alibaba-Zhejiang University Joint Institute of Frontier Technologies, Hangzhou, China
6.Cyber and Big Data Research Laboratory, Edinburgh Napier University, Edinburgh EH11 4BN, U.K.
Recommended Citation
GB/T 7714
Fangzhou Xiong,Zhiyong Liu,Kaizhu Huang,et al. Encoding Primitives Generation Policy Learning for Robotic Arm to Overcome Catastrophic Forgetting in Sequential Multi-tasks Learning[J]. Neural Networks,2020(2020.06.003):12.
APA Fangzhou Xiong,Zhiyong Liu,Kaizhu Huang,Xu Yang,Hong Qiao,&Amir Hussain.(2020).Encoding Primitives Generation Policy Learning for Robotic Arm to Overcome Catastrophic Forgetting in Sequential Multi-tasks Learning.Neural Networks(2020.06.003),12.
MLA Fangzhou Xiong,et al."Encoding Primitives Generation Policy Learning for Robotic Arm to Overcome Catastrophic Forgetting in Sequential Multi-tasks Learning".Neural Networks .2020.06.003(2020):12.
Files in This Item: Download All
File Name/Size DocType Version Access License
Manuscript.pdf(646KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Fangzhou Xiong]'s Articles
[Zhiyong Liu]'s Articles
[Kaizhu Huang]'s Articles
Baidu academic
Similar articles in Baidu academic
[Fangzhou Xiong]'s Articles
[Zhiyong Liu]'s Articles
[Kaizhu Huang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Fangzhou Xiong]'s Articles
[Zhiyong Liu]'s Articles
[Kaizhu Huang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: Manuscript.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.