Diverse Neuron Type Selection for Convolutional Neural Networks | |
Guibo Zhu1; Zhaoxiang Zhang1,2,3; Xu-Yao Zhang1; Cheng-Lin Liu1,2,3 | |
2017 | |
会议名称 | International Joint Conference on Artificial Intelligence |
会议日期 | August 19-25, 2017 |
会议地点 | Australia |
摘要 |
The activation function for neurons is a prominent element in the deep learning architecture for obtaining high performance. Inspired by neuroscience findings, we introduce and define two types of neurons with different activation functions for artificial neural networks: excitatory and inhibitory neurons, which can be adaptively selected by selflearning. Based on the definition of neurons, in the paper we not only unify the mainstream activation functions, but also discuss the complementariness among these types of neurons. In addition, through the cooperation of excitatory and inhibitory neurons, we present a compositional activation function that leads to new state-of-the-art performance comparing to rectifier linear units. Finally, we hope that our framework not only gives a basic unified framework of the existing activation neurons to provide guidance for future design, but also contributes neurobiological explanations which can be treated as a window to bridge the gap between biology and computer science. |
收录类别 | EI |
文献类型 | 会议论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/20453 |
专题 | 智能感知与计算研究中心 |
通讯作者 | Zhaoxiang Zhang |
作者单位 | 1.Institute of Automation, Chinese Academy of Sciences 2.University of Chinese Academy of Sciences 3.Center for Excellence in Brain Science and Intelligence Technology, CAS |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Guibo Zhu,Zhaoxiang Zhang,Xu-Yao Zhang,et al. Diverse Neuron Type Selection for Convolutional Neural Networks[C],2017. |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
diverse neuron type (259KB) | 会议论文 | 开放获取 | CC BY-NC-SA | 浏览 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论