Knowledge Commons of Institute of Automation,CAS
Sparser spiking activity can be better: Feature Refine-and-Mask spiking neural network for event-based visual recognition | |
Yao, Man1,2; Zhang, Hengyu1,3; Zhao, Guangshe1; Zhang, Xiyu1; Wang, Dingheng4; Cao, Gang5; Li, Guoqi2,6 | |
发表期刊 | NEURAL NETWORKS |
ISSN | 0893-6080 |
2023-09-01 | |
卷号 | 166页码:410-423 |
通讯作者 | Zhao, Guangshe(zhaogs@mail.xjtu.edu.cn) ; Li, Guoqi(guoqi.li@ia.ac.cn) |
摘要 | Event-based visual, a new visual paradigm with bio-inspired dynamic perception and & mu;s level temporal resolution, has prominent advantages in many specific visual scenarios and gained much research interest. Spiking neural network (SNN) is naturally suitable for dealing with event streams due to its temporal information processing capability and event-driven nature. However, existing works SNN neglect the fact that the input event streams are spatially sparse and temporally non-uniform, and just treat these variant inputs equally. This situation interferes with the effectiveness and efficiency of existing SNNs. In this paper, we propose the feature Refine-and-Mask SNN (RM-SNN), which has the ability of self-adaption to regulate the spiking response in a data-dependent way. We use the Refine-and-Mask (RM) module to refine all features and mask the unimportant features to optimize the membrane potential of spiking neurons, which in turn drops the spiking activity. Inspired by the fact that not all events in spatio-temporal streams are task-relevant, we execute the RM module in both temporal and channel dimensions. Extensive experiments on seven event-based benchmarks, DVS128 Gesture, DVS128 Gait, CIFAR10-DVS, N-Caltech101, DailyAction-DVS, UCF101-DVS, and HMDB51-DVS demonstrate that under the multi-scale constraints of input time window, RM-SNN can significantly reduce the network average spiking activity rate while improving the task performance. In addition, by visualizing spiking responses, we analyze why sparser spiking activity can be better. Code & COPY; 2023 Elsevier Ltd. All rights reserved. |
关键词 | Spiking neural network Event-based vision Neuromorphic computing Attention mechanism Brain-inspired computing |
DOI | 10.1016/j.neunet.2023.07.008 |
关键词[WOS] | INTELLIGENCE ; DEEPER |
收录类别 | SCI |
语种 | 英语 |
资助项目 | National Natural Science Foundation of China[61836004] ; National Natural Science Foundation of China[62236009] ; National Natural Science Foundation of China[U22A20103] ; National Key Ramp;D Program of China[2020AAA0105200] ; Beijing Natural Science Foundation for Distinguished Young Scholars[JQ21015] ; Pengcheng Lab |
项目资助者 | National Natural Science Foundation of China ; National Key Ramp;D Program of China ; Beijing Natural Science Foundation for Distinguished Young Scholars ; Pengcheng Lab |
WOS研究方向 | Computer Science ; Neurosciences & Neurology |
WOS类目 | Computer Science, Artificial Intelligence ; Neurosciences |
WOS记录号 | WOS:001070932700001 |
出版者 | PERGAMON-ELSEVIER SCIENCE LTD |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/53138 |
专题 | 脑图谱与类脑智能实验室 |
通讯作者 | Zhao, Guangshe; Li, Guoqi |
作者单位 | 1.Xi An Jiao Tong Univ, Sch Automat Sci & Engn, Xian 710049, Shaanxi, Peoples R China 2.Peng Cheng Lab, Shenzhen 518000, Peoples R China 3.Tsinghua Univ, Tsinghua Shenzhen Int Grad Sch, Shenzhen 518000, Peoples R China 4.Northwest Inst Mech & Elect Engn, Xianyang, Shaanxi, Peoples R China 5.Beijing Acad Artificial Intelligence, Beijing 100089, Peoples R China 6.Chinese Acad Sci, Inst Automat, Beijing 100089, Peoples R China |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Yao, Man,Zhang, Hengyu,Zhao, Guangshe,et al. Sparser spiking activity can be better: Feature Refine-and-Mask spiking neural network for event-based visual recognition[J]. NEURAL NETWORKS,2023,166:410-423. |
APA | Yao, Man.,Zhang, Hengyu.,Zhao, Guangshe.,Zhang, Xiyu.,Wang, Dingheng.,...&Li, Guoqi.(2023).Sparser spiking activity can be better: Feature Refine-and-Mask spiking neural network for event-based visual recognition.NEURAL NETWORKS,166,410-423. |
MLA | Yao, Man,et al."Sparser spiking activity can be better: Feature Refine-and-Mask spiking neural network for event-based visual recognition".NEURAL NETWORKS 166(2023):410-423. |
条目包含的文件 | 条目无相关文件。 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论