CASIA OpenIR  > 数字内容技术与服务研究中心  > 听觉模型与认知计算
Deep Neural Networks with Multistate Activation Functions
Cai, Chenghao1; Xu, Yanyan2; Ke, Dengfeng3; Su, Kaile4
AbstractWe propose multistate activation functions (MSAFs) for deep neural networks (DNNs). These MSAFs are new kinds of activation functions which are capable of representingmore than two states, including the N-order MSAFs and the symmetrical MSAF. DNNs with these MSAFs can be trained via conventional Stochastic Gradient Descent (SGD) as well as mean-normalised SGD. We also discuss how theseMSAFs perform when used to resolve classification problems. Experimental results on the TIMIT corpus reveal that, on speech recognition tasks, DNNs withMSAFs perform better than the conventional DNNs, getting a relative improvement of 5.60% on phoneme error rates. Further experiments also reveal that mean-normalised SGD facilitates the training processes of DNNs with MSAFs, especially when being with large training sets. The models can also be directly trained without pretraining when the training set is sufficiently large, which results in a considerable relative improvement of 5.82% on word error rates.
KeywordDeep Learning Speech Recognition
WOS HeadingsScience & Technology ; Life Sciences & Biomedicine
Indexed BySCI
WOS Research AreaMathematical & Computational Biology ; Neurosciences & Neurology
WOS SubjectMathematical & Computational Biology ; Neurosciences
WOS IDWOS:000361694800001
Citation statistics
Cited Times:1[WOS]   [WOS Record]     [Related Records in WOS]
Document Type期刊论文
Affiliation1.Beijing Forestry Univ, Sch Technol, Beijing 100083, Peoples R China
2.Beijing Forestry Univ, Sch Informat Sci & Technol, Beijing 100083, Peoples R China
3.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China
4.Zhejiang Normal Univ, Coll Math Phys & Informat Engn, Jinhua 321004, Peoples R China
Recommended Citation
GB/T 7714
Cai, Chenghao,Xu, Yanyan,Ke, Dengfeng,et al. Deep Neural Networks with Multistate Activation Functions[J]. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE,2015(90):1-10.
APA Cai, Chenghao,Xu, Yanyan,Ke, Dengfeng,&Su, Kaile.(2015).Deep Neural Networks with Multistate Activation Functions.COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE(90),1-10.
MLA Cai, Chenghao,et al."Deep Neural Networks with Multistate Activation Functions".COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE .90(2015):1-10.
Files in This Item: Download All
File Name/Size DocType Version Access License
721367.pdf(2155KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Cai, Chenghao]'s Articles
[Xu, Yanyan]'s Articles
[Ke, Dengfeng]'s Articles
Baidu academic
Similar articles in Baidu academic
[Cai, Chenghao]'s Articles
[Xu, Yanyan]'s Articles
[Ke, Dengfeng]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Cai, Chenghao]'s Articles
[Xu, Yanyan]'s Articles
[Ke, Dengfeng]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: 721367.pdf
Format: Adobe PDF
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.