Machine translation, which acts as an alternative to human-engineered translation, has achieved more and more attention. However, traditional methods based on statistical machine translation (SMT) face serious challenges due to the disadvantages of SMT, such as linear non-separable problems, the lack of global context information, semantic-independent, and error propagation. In recent years, the neural network model based on deep learning technology has achieved remarkable results in many domains, which draws more and more attention of academia and industry and provides a new solution for the bottleneck of machine translation. In most of the previous work, neural network model takes a role in either the key module of SMT or producing end-to-end neural machine translation model to develop the performance of machine translation. In this paper, we focus on applying neural network model in machine translation and the main contributions are as follows:
A bilingual Chinese word segmentation (CWS) method based on cascaded log-linear model is proposed, which involves learning three levels of features including monolingual grammars, bilingual alignment feature, bilingual semantic feature and bilingual transliteration feature. The proposed method guarantees not only the monolingual grammars, but also the low perplexity of bilingual alignment.
A quality evaluation for bilingual parallel corpus based on perplexity computation using neural network model is proposed. Different from traditional methods which suffer from manually heuristic features, lack of global context information and semantic-independent, the proposed method doesn’t have to make any context-free hypothesis and heuristic features. Moreover, neural network model can integrate semantic information of bilingual words, which not only address the problem of synonym, but also deal with transferred meaning in Chinese-English corpus.
A syntactic-constrained hierarchical translation model based on bilingually-constrained recursive neural networks is proposed, which provides two types of syntactic information for standard hierarchical translation model. Different from traditional syntactic-based methods, the proposed method can leverage both syntactic knowledge of source parsing and shallow parsing. Moreover, the proposed method employ a significantly weaker constraint to integrate these two syntactic knowledge, which can alleviate data sparseness and the influence of parsing errors.
A bilingual named entity alignment method based on attention mechanism is proposed. Different from traditional methods, the proposed method can leverage global context information and alleviate the underestimate of probability led by maximum likelihood estimation, which makes it extract bilingual named entity more accurately.
An end-to-end neural machine translation system is built. We integrate Asynchronous stochastic gradient descent (ASGD) algorithm and hierarchical decomposition algorithm to address the training efficiency problem. Moreover, we leverage the researches above to develop the performance of the system. Finally, we evaluate the translation performance of the end-to-end neural machine translation system on multi-domain testing data.