Deep Neural Networks with Multistate Activation Functions
Cai, Chenghao1; Xu, Yanyan2; Ke, Dengfeng3; Su, Kaile4
发表期刊COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE
2015
期号90页码:1-10
文章类型Article
摘要We propose multistate activation functions (MSAFs) for deep neural networks (DNNs). These MSAFs are new kinds of activation functions which are capable of representingmore than two states, including the N-order MSAFs and the symmetrical MSAF. DNNs with these MSAFs can be trained via conventional Stochastic Gradient Descent (SGD) as well as mean-normalised SGD. We also discuss how theseMSAFs perform when used to resolve classification problems. Experimental results on the TIMIT corpus reveal that, on speech recognition tasks, DNNs withMSAFs perform better than the conventional DNNs, getting a relative improvement of 5.60% on phoneme error rates. Further experiments also reveal that mean-normalised SGD facilitates the training processes of DNNs with MSAFs, especially when being with large training sets. The models can also be directly trained without pretraining when the training set is sufficiently large, which results in a considerable relative improvement of 5.82% on word error rates.
关键词Deep Learning Speech Recognition
WOS标题词Science & Technology ; Life Sciences & Biomedicine
DOI10.1155/2015/721367
关键词[WOS]GRADIENT DESCENT
收录类别SCI
语种英语
WOS研究方向Mathematical & Computational Biology ; Neurosciences & Neurology
WOS类目Mathematical & Computational Biology ; Neurosciences
WOS记录号WOS:000361694800001
引用统计
被引频次:1[WOS]   [WOS记录]     [WOS相关记录]
文献类型期刊论文
条目标识符http://ir.ia.ac.cn/handle/173211/10735
专题数字内容技术与服务研究中心_听觉模型与认知计算
作者单位1.Beijing Forestry Univ, Sch Technol, Beijing 100083, Peoples R China
2.Beijing Forestry Univ, Sch Informat Sci & Technol, Beijing 100083, Peoples R China
3.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China
4.Zhejiang Normal Univ, Coll Math Phys & Informat Engn, Jinhua 321004, Peoples R China
推荐引用方式
GB/T 7714
Cai, Chenghao,Xu, Yanyan,Ke, Dengfeng,et al. Deep Neural Networks with Multistate Activation Functions[J]. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE,2015(90):1-10.
APA Cai, Chenghao,Xu, Yanyan,Ke, Dengfeng,&Su, Kaile.(2015).Deep Neural Networks with Multistate Activation Functions.COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE(90),1-10.
MLA Cai, Chenghao,et al."Deep Neural Networks with Multistate Activation Functions".COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE .90(2015):1-10.
条目包含的文件 下载所有文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
721367.pdf(2155KB)期刊论文作者接受稿开放获取CC BY-NC-SA浏览 下载
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[Cai, Chenghao]的文章
[Xu, Yanyan]的文章
[Ke, Dengfeng]的文章
百度学术
百度学术中相似的文章
[Cai, Chenghao]的文章
[Xu, Yanyan]的文章
[Ke, Dengfeng]的文章
必应学术
必应学术中相似的文章
[Cai, Chenghao]的文章
[Xu, Yanyan]的文章
[Ke, Dengfeng]的文章
相关权益政策
暂无数据
收藏/分享
文件名: 721367.pdf
格式: Adobe PDF
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。