CASIA OpenIR  > 模式识别国家重点实验室  > 自然语言处理
A Substitution-Translation-Restoration Framework for Handling Unknown Words in Statistical Machine Translation
Zhang, Jiajun; Zhai, Feifei; Zong, Chengqing
Source PublicationJournal of Computer Science and Technology
2013-09
Volume28Issue:5Pages:907-918
AbstractUnknown words are one of the key factors that greatly affect the translation quality. Traditionally, nearly all the related research work focus on obtaining the translation of the unknown words. However, these approaches have two disadvantages. On the one hand, they usually rely on many additional resources such as bilingual web data; on the other hand, they cannot guarantee good reordering and lexical selection of surrounding words. This paper gives a new perspective on handling unknown words in statistical machine translation (SMT). Instead of making great efforts to find the translation of unknown words, we focus on determining the semantic function of the unknown words in the test sentence and keeping the semantic function unchanged in the translation process. In this way, unknown words can help the phrase reordering and lexical selection of their surrounding words even though they still remain untranslated. In order to determine the semantic function of an unknown word, we employ the distributional semantic model and the bidirectional language model. Extensive experiments on both phrase-based and linguistically syntax-based SMT models in Chinese-to-English translation show that our methods can substantially improve the translation quality.
Other Abstract未登录词是影响译文质量的关键因素之一。传统上,几乎所有的相关研究工作都集中在未知词的翻译上。然而,这些方法有两个缺点。一方面,它们通常依赖于许多附加资源,如双语Web数据;另一方面,它们不能保证周围词语的重新排序和词汇选择。本文提出了统计机器翻译(SMT)中未知词处理的新视角。而不是努力寻找未知词的翻译,我们的重点是确定在测试句子中的未知词的语义功能,保持在翻译过程中的语义功能不变。这样,未知的话,可以帮助短语排序和其周围的词汇选择即使他们仍然未翻译。为了确定一个未知词的语义功能,我们采用了分布式语义模型和双向语言模型。基于短语和语言句法的SMT模型在汉英翻译中的大量实验表明,我们的方法可以大大提高翻译质量。
KeywordStatistical Machine Translation Distributional Semantics Bidirectional Language Model
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/20686
Collection模式识别国家重点实验室_自然语言处理
Recommended Citation
GB/T 7714
Zhang, Jiajun,Zhai, Feifei,Zong, Chengqing. A Substitution-Translation-Restoration Framework for Handling Unknown Words in Statistical Machine Translation[J]. Journal of Computer Science and Technology,2013,28(5):907-918.
APA Zhang, Jiajun,Zhai, Feifei,&Zong, Chengqing.(2013).A Substitution-Translation-Restoration Framework for Handling Unknown Words in Statistical Machine Translation.Journal of Computer Science and Technology,28(5),907-918.
MLA Zhang, Jiajun,et al."A Substitution-Translation-Restoration Framework for Handling Unknown Words in Statistical Machine Translation".Journal of Computer Science and Technology 28.5(2013):907-918.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Zhang, Jiajun]'s Articles
[Zhai, Feifei]'s Articles
[Zong, Chengqing]'s Articles
Baidu academic
Similar articles in Baidu academic
[Zhang, Jiajun]'s Articles
[Zhai, Feifei]'s Articles
[Zong, Chengqing]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Zhang, Jiajun]'s Articles
[Zhai, Feifei]'s Articles
[Zong, Chengqing]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.