CASIA OpenIR  > 脑图谱与类脑智能实验室  > 类脑认知计算
MSAT: biologically inspired multistage adaptive threshold for conversion of spiking neural networks
He, Xiang1,2,3; Li, Yang1,2,3; Zhao, Dongcheng1,2; Kong, Qingqun1,2,3,4; Zeng, Yi1,2,3,4,5
Source PublicationNeural Computing and Applications
2024
Pages8531–8547
Abstract

Spiking neural networks (SNNs) can do inference with low power consumption due to their spike sparsity. Although SNNs can be combined with neuromorphic hardware to achieve efficient inference, they are often difficult to train directly due to discrete non-differentiable spikes. As an alternative, ANN-SNN conversion is an efficient way to achieve deep SNNs by converting well-trained artificial neural networks (ANNs). However, the existing methods commonly use constant threshold for conversion. A high constant threshold value prevents neurons from rapidly delivering spikes to deeper layers and causes high time delay. In addition, the same response for different inputs may result in information loss during the information transmission. Inspired by the biological adaptive threshold mechanism, we propose a multistage adaptive threshold (MSAT) method to alleviate this problem. Instead of using a single, constant value, the threshold is adjusted in multistages, adapting to each neuron’s firing history and input properties. Specifically, for each neuron, the dynamic threshold is positively correlated with the average membrane potential and negatively correlated with the rate of depolarization. The adaptation to membrane potential and input allows a timely adjustment of the threshold to fire spikes faster and transmit more information. Moreover, we analyze the spikes of inactivated neurons error, which is pervasive in early time steps. We also propose spike confidence accordingly to measure confidence about the neurons that correctly deliver spikes. Such spike confidence in early time steps is used to determine whether to elicit the spike to alleviate the spikes of inactivated neurons error. Combined with the proposed methods, we examine the performance on CIFAR-10, CIFAR-100, and ImageNet datasets. We also conduct sentiment classification and speech recognition experiments on the IDBM and Google speech commands datasets, respectively. Experiments show that our methods can achieve near-lossless and lower latency ANN-SNN conversion. In summary, we build a biologically inspired multistage adaptive threshold for converted SNN, with comparable performance to state-of-the-art methods while improving energy efficiency.

MOST Discipline Catalogue工学::控制科学与工程
DOIhttps://doi.org/10.1007/s00521-024-09529-w
URL查看原文
Indexed BySCI
Sub direction classification类脑模型与计算
planning direction of the national heavy laboratory认知机理与类脑学习
Paper associated data
Citation statistics
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/57242
Collection脑图谱与类脑智能实验室_类脑认知计算
Corresponding AuthorKong, Qingqun; Zeng, Yi
Affiliation1.Brain-inspired Cognitive Intelligence Lab, Chinese Academy of Sciences, Beijing, China
2.Center for Long-term Artificial Intelligence, Beijing, China
3.School of Artificial Intelligence, Chinese Academy of Sciences, Beijing, China
4.School of Future Technology, Chinese Academy of Sciences, Beijing, China
5.Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Chinese Academy of Sciences, Shanghai, China
Recommended Citation
GB/T 7714
He, Xiang,Li, Yang,Zhao, Dongcheng,et al. MSAT: biologically inspired multistage adaptive threshold for conversion of spiking neural networks[J]. Neural Computing and Applications,2024:8531–8547.
APA He, Xiang,Li, Yang,Zhao, Dongcheng,Kong, Qingqun,&Zeng, Yi.(2024).MSAT: biologically inspired multistage adaptive threshold for conversion of spiking neural networks.Neural Computing and Applications,8531–8547.
MLA He, Xiang,et al."MSAT: biologically inspired multistage adaptive threshold for conversion of spiking neural networks".Neural Computing and Applications (2024):8531–8547.
Files in This Item: Download All
File Name/Size DocType Version Access License
NCAA_MSAT.pdf(2459KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[He, Xiang]'s Articles
[Li, Yang]'s Articles
[Zhao, Dongcheng]'s Articles
Baidu academic
Similar articles in Baidu academic
[He, Xiang]'s Articles
[Li, Yang]'s Articles
[Zhao, Dongcheng]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[He, Xiang]'s Articles
[Li, Yang]'s Articles
[Zhao, Dongcheng]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: NCAA_MSAT.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.