Multilingual Recurrent Neural Networks with Residual Learning for Low-Resource Speech Recognition

	Multilingual Recurrent Neural Networks with Residual Learning for Low-Resource Speech Recognition
	Shiyu Zhou; Yuanyuan Zhao; Shuang Xu; Bo Xu
	2017
会议名称	Interspeech
会议录名称	Interspeech
会议日期	2017
会议地点	Stockholm
摘要	The shared-hidden-layer multilingual deep neural network (SHL-MDNN), in which the hidden layers of feed-forward deep neural network (DNN) are shared across multiple languages while the softmax layers are language dependent, has been shown to be effective on acoustic modeling of multilingual low-resource speech recognition. In this paper, we propose that the shared-hidden-layer with Long Short-Term Memory (LSTM) recurrent neural networks can achieve further performance improvement considering LSTM has outperformed DNN as the acoustic model of automatic speech recognition (ASR). Moreover, we reveal that shared-hidden-layer multilingual LSTM (SHL-MLSTM) with residual learning can yield additional moderate but consistent gain from multilingual tasks given the fact that residual learning can allievate the degradation problem of deep LSTMs. Experimental results demonstrate that SHL-MLSTM can relatively reduce word error rate (WER) by 2.1-6.8\% over SHL-MDNN trained using six languages and 2.6-7.3\% over monolingual LSTM trained using the language specific data on CALLHOME datasets. Additional WER reduction, about relatively 2\% over SHL-MLSTM, can be obtained through residual learning on CALLHOME datasets, which demonstrates residual learning is useful for SHL-MLSTM on multilingual low-resource ASR.
关键词	Lstm Multilingual Speech Recognition Low-resource Residual Learning Shared-hidden-layer
收录类别	EI
语种	英语
文献类型	会议论文
条目标识符	http://ir.ia.ac.cn/handle/173211/41058
专题	复杂系统认知与决策实验室_听觉模型与认知计算
通讯作者	Shiyu Zhou
推荐引用方式 GB/T 7714	Shiyu Zhou,Yuanyuan Zhao,Shuang Xu,et al. Multilingual Recurrent Neural Networks with Residual Learning for Low-Resource Speech Recognition[C],2017.