CASIA OpenIR  > 数字内容技术与服务研究中心  > 听觉模型与认知计算
ASYNCHRONOUS STOCHASTIC GRADIENT DESCENT FOR DNN TRAINING
Shanshan, Zhang; Ce, Zhang; Zhao, You; Rong, Zheng; Bo, Xu
2013
Conference Name2013 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP)
Source PublicationIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Conference Date2013
Conference PlaceVancouver, Canada
Abstract
; It is well known that state-of-the-art speech recognition systems
using deep neural network (DNN) can greatly improve the system
performance compared with conventional GMM-HMM. However,
what we have to pay correspondingly is the immense training cost
due to the enormous parameters of DNN. Unfortunately, it is difficult
to achieve parallelization of the minibatch-based back-propagation
(BP) algorithm used in DNN training because of the frequent model
updates.
In this paper we describe an effective approach to achieve an
approximation of BP — asynchronous stochastic gradient descent
(ASGD), which is used to parallelize computing on multi-GPU. This
approach manages multiple GPUs to work asynchronously to calculate
gradients and update the global model parameters. Experimental
results show that it achieves a 3.2 times speed-up on 4 GPUs than the
single one, without any recognition performance loss.
KeywordDeep Neural Network Speech Recognition Asynchronous Sgd Gpu Parallelization
Indexed ByEI
Language英语
Document Type会议论文
Identifierhttp://ir.ia.ac.cn/handle/173211/11808
Collection数字内容技术与服务研究中心_听觉模型与认知计算
Corresponding AuthorShanshan, Zhang
AffiliationInteractive Digital Media Technology Research Center Institute of Automation, Chinese Academy of Sciences
First Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Corresponding Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Shanshan, Zhang,Ce, Zhang,Zhao, You,et al. ASYNCHRONOUS STOCHASTIC GRADIENT DESCENT FOR DNN TRAINING[C],2013.
Files in This Item: Download All
File Name/Size DocType Version Access License
20121201085547_33344(260KB)会议论文 开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Shanshan, Zhang]'s Articles
[Ce, Zhang]'s Articles
[Zhao, You]'s Articles
Baidu academic
Similar articles in Baidu academic
[Shanshan, Zhang]'s Articles
[Ce, Zhang]'s Articles
[Zhao, You]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Shanshan, Zhang]'s Articles
[Ce, Zhang]'s Articles
[Zhao, You]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: 20121201085547_333442_3735.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.