Continual learning of context-dependent processing in neural networks

	Continual learning of context-dependent processing in neural networks
	Ceng GX(曾冠雄); Chen Y(陈阳); Cui B(崔波); Yu S(余山)
发表期刊	nature machine intelligence
	2019-08
期号	1 页码:364–372
摘要	Deep neural networks are powerful tools in learning sophisticated but fixed mapping rules between inputs and outputs, thereby limiting their application in more complex and dynamic situations in which the mapping rules are not kept the same but change according to different contexts. To lift such limits, we developed an approach involving a learning algorithm, called orthogonal weights modification, with the addition of a context-dependent processing module. We demonstrated that with orthogonal weights modification to overcome catastrophic forgetting, and the context-dependent processing module to learn how to reuse a feature representation and a classifier for different contexts, a single network could acquire numerous context-dependent mapping rules in an online and continual manner, with as few as approximately ten samples to learn each. Our approach should enable highly compact systems to gradually learn myriad regularities of the real world and eventually behave appropriately within it.
关键词	Continual Learning，Context dependent Learning
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/26152
专题	脑图谱与类脑智能实验室_脑网络组研究
通讯作者	Yu S(余山)
推荐引用方式 GB/T 7714	Ceng GX,Chen Y,Cui B,et al. Continual learning of context-dependent processing in neural networks[J]. nature machine intelligence,2019(1):364–372.
APA	Ceng GX,Chen Y,Cui B,&Yu S.(2019).Continual learning of context-dependent processing in neural networks.nature machine intelligence(1),364–372.
MLA	Ceng GX,et al."Continual learning of context-dependent processing in neural networks".nature machine intelligence .1(2019):364–372.