CASIA OpenIR  > 数字内容技术与服务研究中心  > 听觉模型与认知计算
ASYNCHRONOUS STOCHASTIC GRADIENT DESCENT FOR DNN TRAINING
Shanshan, Zhang; Ce, Zhang; Zhao, You; Rong, Zheng; Bo, Xu
2013
Conference Name2013 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP)
Source PublicationIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Conference Date2013
Conference PlaceVancouver, Canada
Abstract
; It is well known that state-of-the-art speech recognition systems
using deep neural network (DNN) can greatly improve the system
performance compared with conventional GMM-HMM. However,
what we have to pay correspondingly is the immense training cost
due to the enormous parameters of DNN. Unfortunately, it is difficult
to achieve parallelization of the minibatch-based back-propagation
(BP) algorithm used in DNN training because of the frequent model
updates.
In this paper we describe an effective approach to achieve an
approximation of BP — asynchronous stochastic gradient descent
(ASGD), which is used to parallelize computing on multi-GPU. This
approach manages multiple GPUs to work asynchronously to calculate
gradients and update the global model parameters. Experimental
results show that it achieves a 3.2 times speed-up on 4 GPUs than the
single one, without any recognition performance loss.
KeywordDeep Neural Network Speech Recognition Asynchronous Sgd Gpu Parallelization
Indexed ByEI
Language英语
Document Type会议论文
Identifierhttp://ir.ia.ac.cn/handle/173211/11808
Collection数字内容技术与服务研究中心_听觉模型与认知计算
Corresponding AuthorShanshan, Zhang
AffiliationInteractive Digital Media Technology Research Center Institute of Automation, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Shanshan, Zhang,Ce, Zhang,Zhao, You,et al. ASYNCHRONOUS STOCHASTIC GRADIENT DESCENT FOR DNN TRAINING[C],2013.
Files in This Item: Download All
File Name/Size DocType Version Access License
20121201085547_33344(260KB)会议论文 开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Shanshan, Zhang]'s Articles
[Ce, Zhang]'s Articles
[Zhao, You]'s Articles
Baidu academic
Similar articles in Baidu academic
[Shanshan, Zhang]'s Articles
[Ce, Zhang]'s Articles
[Zhao, You]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Shanshan, Zhang]'s Articles
[Ce, Zhang]'s Articles
[Zhao, You]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: 20121201085547_333442_3735.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.