Knowledge Commons of Institute of Automation,CAS
Solving the spike feature information vanishing problem in spiking deep Q network with potential based normalization | |
Sun, Yinqian1,2; Zeng, Yi1,2,3,4,5; Li, Yang1,3 | |
发表期刊 | FRONTIERS IN NEUROSCIENCE |
2022-08-25 | |
卷号 | 16页码:11 |
摘要 | Brain-inspired spiking neural networks (SNNs) are successfully applied to many pattern recognition domains. The SNNs-based deep structure has achieved considerable results in perceptual tasks, such as image classification and target detection. However, applying deep SNNs in reinforcement learning (RL) tasks is still a problem to be explored. Although there have been previous studies on the combination of SNNs and RL, most focus on robotic control problems with shallow networks or using the ANN-SNN conversion method to implement spiking deep Q networks (SDQN). In this study, we mathematically analyzed the problem of the disappearance of spiking signal features in SDQN and proposed a potential-based layer normalization (pbLN) method to train spiking deep Q networks directly. Experiment shows that compared with state-of-art ANN-SNN conversion method and other SDQN works, the proposed pbLN spiking deep Q networks (PL-SDQN) achieved better performance on Atari game tasks. |
关键词 | brain-inspired decision model SDQN reinforcement learning potential normalization spiking activity |
DOI | 10.3389/fnins.2022.953368 |
收录类别 | SCI |
语种 | 英语 |
资助项目 | National Key Research and Development Program ; Strategic Priority Research Program of the Chinese Academy of Sciences ; National Natural Science Foundation of China ; [2020AAA0104305] ; [XDB32070100] ; [62106261] |
项目资助者 | National Key Research and Development Program ; Strategic Priority Research Program of the Chinese Academy of Sciences ; National Natural Science Foundation of China |
WOS研究方向 | Neurosciences & Neurology |
WOS类目 | Neurosciences |
WOS记录号 | WOS:000852629100001 |
出版者 | FRONTIERS MEDIA SA |
七大方向——子方向分类 | 类脑模型与计算 |
国重实验室规划方向分类 | 其他 |
是否有论文关联数据集需要存交 | 否 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/50084 |
专题 | 脑图谱与类脑智能实验室_类脑认知计算 |
通讯作者 | Zeng, Yi |
作者单位 | 1.Chinese Acad Sci, Inst Automat, Res Ctr Brain Inspired Intelligence, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Sch Future Technol, Beijing, Peoples R China 3.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing, Peoples R China 4.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing, Peoples R China 5.Chinese Acad Sci, Ctr Excellence Brain Sci & Intelligence Technol, Shanghai, Peoples R China |
第一作者单位 | 类脑智能研究中心 |
通讯作者单位 | 类脑智能研究中心; 模式识别国家重点实验室 |
推荐引用方式 GB/T 7714 | Sun, Yinqian,Zeng, Yi,Li, Yang. Solving the spike feature information vanishing problem in spiking deep Q network with potential based normalization[J]. FRONTIERS IN NEUROSCIENCE,2022,16:11. |
APA | Sun, Yinqian,Zeng, Yi,&Li, Yang.(2022).Solving the spike feature information vanishing problem in spiking deep Q network with potential based normalization.FRONTIERS IN NEUROSCIENCE,16,11. |
MLA | Sun, Yinqian,et al."Solving the spike feature information vanishing problem in spiking deep Q network with potential based normalization".FRONTIERS IN NEUROSCIENCE 16(2022):11. |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
fnins-16-953368.pdf(1561KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论