毕业生知识库

（2）经典的跨语言参数迁移方法存在两方面的不足：一是忽略了多语言模型应

学习语言相关特征这一事实；二是共享隐层学习了很多语言依赖的特征。为了弥

补这些缺陷，本文提出了对语言对抗的模型参数迁移方法。本文将语言对抗策略

与迁移学习方法相结合训练对抗共享私有模型，此外，提出了两种新的迁移策略。

共享私有模型不仅能学习语言无关的特征，而且能捕捉语言依赖的信息。语言对

抗策略保证了共享层尽可能多地学习通用特征。语言无关的通用特征能显著地提

高目标声学模型的性能。在IARPA Babel 数据集上的实验结果表明，基于语言对

抗学习的模型参数迁移方法与经典的跨语言参数迁移方法相比，词错误率最多相

对下降了9.7%。

（3）若直接对基于联结时序分类（connectionist temporal classification，CTC）

Deep neural network based acoustic models have obtained significant improvement for automatic speech recognition (ASR) systems. However, deep neural network are highly effective when trained with large amounts of transcribed speech data. While such annotations may not be readily available for most languages. Moreover, resource-rich language like Mandarin also has annotation sparsity problem when ASR systems deal with different accented speech. Without any question, data collection and annotation are much time and money-consuming. Therefore, it is valuable and challenging to address this problem. The goal of this thesis is to use transfer learning to address the challenges for low-resource deep neural network based acoutic models. In other words, this thesis uses transfer learning to tranfer the knowledge from other languages to the novel target language. In particular, this thesis focuses on two transfer learning scenarios: transferring across different languages and transferring cross accents for the same language. This thesis proposes three novel transfer learning approaches to improve the performance of the low-resource language. The main contributions of this thesis are as follows.

(1) Multilingual bottleneck features are helpful to improve the perfomance of low-resource speech recognition systems. However, this method has some shortcomings. One is that the relevance of the source and the target languages is ignored. The other is that the bottleneck features may contain some unnecessary language specific information. This thesis proposes an adversarial multilingual training to alleviate this problem. Adversarial training is used to ensure that the shared layers can learn language invariant features. Moreover, the languages which have the same language family as the target language are selected as the source languages. Experiments are conducted on IARPA Babel datasets. The results show that the proposed adversarial multilingual BN model outperforms the baseline BN model by

up to 8.9% relative word error rate (WER) reduction.

(2) The target acoustic model trained using the cross-lingual parameters transferred from the shared hidden layermodel (SHL-Model) outperforms the model trained only using the target language data, especially when the target language is under low-resource condition. However, the SHL-Model only uses the hidden layers to learn shared features, which ignores the fact that some features are languagespecific. Moreover, the shared features may contain some unnecessary language dependent information. Therefore, this thesis proposes a language-adversarial transfer learning to alleviate this problem. The shared-private source models are proposed to learn both language-dependent and -independent features. Furthermore, adversarial learning is used to ensure that the shared layers of the shared-private model can learn more language-invariant features. Experiments are conducted on IARPA Babel dataset. The results show that the target model trained using the knowledge transferred from the adversarial shared-private model obtains further performance improvement by up to 9.7% relative WER reduction over the target model trained using the knowledge transferred from the SHL-Model.

(3) In general, directly adjusting the network parameters with a small adaptation set may lead to over-fitting. In order to avoid this problem, this thesis proposes a connectionist temporal classification (CTC) regularized adaptation method. The basic idea of this thesis is that a regularization term is added to the CTC training criterion. It forces the conditional probability distribution estimated from the

adapted model to be close to the accent independent model. In other words, the probability distribution is transferred from the accent independent model to the adapted model. Experiments are conducted on RASC863 and CASIA regional accented speech corpus. The results show that the proposed method obtains obvious improvement when compared with the strong baseline model. It also outperforms other adaptation methods, such as L2 and linear hidden network (LHN). 

In addition, we not only propose three novel transfer learning methods for low resource acoustic model, but also apply these methods to real speech recognition systems. In transferring across different languages scenario, we have bulit speech recognition systems for low-resource languages, such as Cantonese, Shanghai dialect and Mongolian etc. In transferring cross accents for the same language scenario, we use the CTC regularized adaptation method to perform accent adaptation for Mandarin acoustic model. The Mandarin speech recognition system is used as an important part of the customer service quality inspection and dialogue system respectively.