CASIA OpenIR  > 模式识别国家重点实验室  > 机器人视觉
Improving Deep Neural Networks Using Softplus Units
Hao Zheng1; Zhanlei Yang1; Wenju Liu1; Jizhong Liang2; Yanpeng Li2
2015
会议名称IJCNN
会议录名称IJCNN
会议日期2015
会议地点Ireland
摘要Recently, DNNs have achieved great improvement for acoustic modeling in speech recognition tasks. However, it is difficult to train the models well when the depth grows. One main reason is that when training DNNs with traditional sigmoid units, the derivatives damp sharply while back-propagating between layers, which restrict the depth of model especially with insufficient training data. To deal with this problem, some unbounded activation functions have been proposed to preserve sufficient gradients, including ReLU and softplus. Compared with ReLU, the smoothing and nonzero properties of the in gradient makes softplus-based DNNs perform better in both stabilization and performance. However, softplus-based DNNs have been rarely
exploited for the phoneme recognition task. In this paper, we explore the use of softplus units for DNNs in acoustic modeling for context-independent phoneme recognition tasks.The revised RBM pre-training and dropout strategy are also applied to improve the performance of softplus units. Experiments show that, the DNNs with softplus units get significantly performance improvement and uses less epochs to get convergence compared to the DNNs trained with standard sigmoid units and ReLUs.
关键词Softplus Dropout Deep Neural Networks Timit
收录类别EI
文献类型会议论文
条目标识符http://ir.ia.ac.cn/handle/173211/11777
专题模式识别国家重点实验室_机器人视觉
通讯作者Hao Zheng
作者单位1.National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences
2.Electric Power Research Institute of Shanxi Electric Power Company
推荐引用方式
GB/T 7714
Hao Zheng,Zhanlei Yang,Wenju Liu,et al. Improving Deep Neural Networks Using Softplus Units[C],2015.
条目包含的文件 下载所有文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
IJCNN-2015-1.pdf(232KB)会议论文 开放获取CC BY-NC-SA浏览 下载
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[Hao Zheng]的文章
[Zhanlei Yang]的文章
[Wenju Liu]的文章
百度学术
百度学术中相似的文章
[Hao Zheng]的文章
[Zhanlei Yang]的文章
[Wenju Liu]的文章
必应学术
必应学术中相似的文章
[Hao Zheng]的文章
[Zhanlei Yang]的文章
[Wenju Liu]的文章
相关权益政策
暂无数据
收藏/分享
文件名: IJCNN-2015-1.pdf
格式: Adobe PDF
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。