Knowledge Commons of Institute of Automation,CAS
Understanding Memory Modules on Learning Simple Algorithms | |
Wang, Kexin1,2; Zhou, Yu1,2; Wang, Shaonan1,2; Zhang, Jiajun1,2; Zong, Chengqing1,2,3 | |
2019-08 | |
会议名称 | International Joint Conferences on Artificial Intelligence 2019 Workshop on Explainable Artificial Intelligence |
会议日期 | 2019-8-11 |
会议地点 | Macau, China |
摘要 | Recent work has shown that memory modules are crucial for the generalization ability of neural networks on learning simple algorithms. However, we still have little understanding of the working mechanism of memory modules. To alleviate this problem, we apply a two-step analysis pipeline consisting of first inferring hypothesis about what strategy the model has learned according to visualization and then verify it by a novel proposed qualitative analysis method based on dimension reduction. Using this method, we have analyzed two popular memory-augmented neural networks, neural Turing machine and stack-augmented neural network on two simple algorithm tasks including reversing a random sequence and evaluation of arithmetic expressions. Results have shown that on the former task both models can learn to generalize and on the latter task only the stack-augmented model can do so. We show that different strategies are learned by the models, in which specific categories of input are monitored and different policies are made based on that to change the memory. |
语种 | 英语 |
七大方向——子方向分类 | 自然语言处理 |
文献类型 | 会议论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/38557 |
专题 | 多模态人工智能系统全国重点实验室_自然语言处理 |
通讯作者 | Wang, Kexin |
作者单位 | 1.National Laboratory of Pattern Recognition, CASIA, Beijing, China 2.University of Chinese Academy of Sciences, Beijing, China 3.CAS Center for Excellence in Brain Science and Intelligence Technology, Beijing, China |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Wang, Kexin,Zhou, Yu,Wang, Shaonan,et al. Understanding Memory Modules on Learning Simple Algorithms[C],2019. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
paper 13.pdf(2643KB) | 会议论文 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论