Overcoming Catastrophic Forgetting with Self-adaptive Identifiers
Fangzhou Xiong1,2; Zhiyong Liu1,2,3,4; Xu Yang1,2
Conference NameInternational Conference on Neural Information Processing
Conference DateDecember 13-16, 2018
Conference PlaceSiem Reap, Cambodia

Catastrophic forgetting is a tough issue when the agent faces the sequential multi-task learning scenario without storing previous task information. It gradually becomes an obstacle to achieve artificial general intelligence which is generally believed to behave like a human with continuous learning capability. In this paper, we propose to utilize the variational Bayesian inference method to overcome catastrophic forgetting. By pruning the neural network according to the mean and variance of weights, parameters are vastly reduced, which mitigates the storage problem of double parameters required in variational Bayesian inference. Based on this lightweight version, autoencoders trained on different tasks are employed to self-adaptively match the corresponding task parameters to tackle sequential multi-task learning problem. We show experimentally on several fundamental datasets that the proposed method can perform substantial improvements without catastrophic forgetting over other classic methods especially in the setting where the probability distributions between tasks present more different.

Document Type会议论文
Corresponding AuthorZhiyong Liu
Affiliation1.State Key Lab of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Science, China
2.School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS), China
3.CAS Centre for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, China
4.Cloud Computing Center, Chinese Academy of Sciences, DongGuan, GuangDong, China
Recommended Citation
GB/T 7714
Fangzhou Xiong,Zhiyong Liu,Xu Yang. Overcoming Catastrophic Forgetting with Self-adaptive Identifiers[C],2018.
Files in This Item: Download All
File Name/Size DocType Version Access License
Overcoming Catastrop(197KB)会议论文 开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Fangzhou Xiong]'s Articles
[Zhiyong Liu]'s Articles
[Xu Yang]'s Articles
Baidu academic
Similar articles in Baidu academic
[Fangzhou Xiong]'s Articles
[Zhiyong Liu]'s Articles
[Xu Yang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Fangzhou Xiong]'s Articles
[Zhiyong Liu]'s Articles
[Xu Yang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: Overcoming Catastrophic Forgetting with Self-adaptive Identifiers.pdf
Format: Adobe PDF
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.