Knowledge Commons of Institute of Automation,CAS
MSAT: biologically inspired multistage adaptive threshold for conversion of spiking neural networks | |
He, Xiang1,2,3; Li, Yang1,2,3![]() ![]() ![]() ![]() | |
发表期刊 | Neural Computing and Applications
![]() |
2024 | |
页码 | 8531–8547 |
摘要 | Spiking neural networks (SNNs) can do inference with low power consumption due to their spike sparsity. Although SNNs can be combined with neuromorphic hardware to achieve efficient inference, they are often difficult to train directly due to discrete non-differentiable spikes. As an alternative, ANN-SNN conversion is an efficient way to achieve deep SNNs by converting well-trained artificial neural networks (ANNs). However, the existing methods commonly use constant threshold for conversion. A high constant threshold value prevents neurons from rapidly delivering spikes to deeper layers and causes high time delay. In addition, the same response for different inputs may result in information loss during the information transmission. Inspired by the biological adaptive threshold mechanism, we propose a multistage adaptive threshold (MSAT) method to alleviate this problem. Instead of using a single, constant value, the threshold is adjusted in multistages, adapting to each neuron’s firing history and input properties. Specifically, for each neuron, the dynamic threshold is positively correlated with the average membrane potential and negatively correlated with the rate of depolarization. The adaptation to membrane potential and input allows a timely adjustment of the threshold to fire spikes faster and transmit more information. Moreover, we analyze the spikes of inactivated neurons error, which is pervasive in early time steps. We also propose spike confidence accordingly to measure confidence about the neurons that correctly deliver spikes. Such spike confidence in early time steps is used to determine whether to elicit the spike to alleviate the spikes of inactivated neurons error. Combined with the proposed methods, we examine the performance on CIFAR-10, CIFAR-100, and ImageNet datasets. We also conduct sentiment classification and speech recognition experiments on the IDBM and Google speech commands datasets, respectively. Experiments show that our methods can achieve near-lossless and lower latency ANN-SNN conversion. In summary, we build a biologically inspired multistage adaptive threshold for converted SNN, with comparable performance to state-of-the-art methods while improving energy efficiency. |
学科门类 | 工学::控制科学与工程 |
DOI | https://doi.org/10.1007/s00521-024-09529-w |
URL | 查看原文 |
收录类别 | SCI |
七大方向——子方向分类 | 类脑模型与计算 |
国重实验室规划方向分类 | 认知机理与类脑学习 |
是否有论文关联数据集需要存交 | 否 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/57242 |
专题 | 脑图谱与类脑智能实验室_类脑认知计算 |
通讯作者 | Kong, Qingqun; Zeng, Yi |
作者单位 | 1.Brain-inspired Cognitive Intelligence Lab, Chinese Academy of Sciences, Beijing, China 2.Center for Long-term Artificial Intelligence, Beijing, China 3.School of Artificial Intelligence, Chinese Academy of Sciences, Beijing, China 4.School of Future Technology, Chinese Academy of Sciences, Beijing, China 5.Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Chinese Academy of Sciences, Shanghai, China |
推荐引用方式 GB/T 7714 | He, Xiang,Li, Yang,Zhao, Dongcheng,et al. MSAT: biologically inspired multistage adaptive threshold for conversion of spiking neural networks[J]. Neural Computing and Applications,2024:8531–8547. |
APA | He, Xiang,Li, Yang,Zhao, Dongcheng,Kong, Qingqun,&Zeng, Yi.(2024).MSAT: biologically inspired multistage adaptive threshold for conversion of spiking neural networks.Neural Computing and Applications,8531–8547. |
MLA | He, Xiang,et al."MSAT: biologically inspired multistage adaptive threshold for conversion of spiking neural networks".Neural Computing and Applications (2024):8531–8547. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
NCAA_MSAT.pdf(2459KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论