Large-scale bilingual parallel corpus is an important resource in many language processing tasks. Especially for statistical machine translation (SMT) system, the size and quality of the training data has a vital impact on its performance. However, the construction of bilingual parallel corpus has been unfulfilling which is far away to be used in real text processing, because of its limitation in size, domain and language pairs. In recent years, with the popularization and rapid development of the Internet, more and more information has been published in multiple languages which make the Internet itself a huge multilingual corpus. Therefore, this thesis is committed to the studies and explorations on the method of mining useful information from the large amount of available resources from the Internet, which is definitely an effective way to solve the problem of bilingual corpus construction and translation knowledge acquisition. Inspired by the above motivation, the main contributions of this thesis can be summarized as follows: 1.A framework to efficiently extract parallel units in two-level is proposed. There is a vast amout of bilingual comparable corpora on the Internet and it’s a big challenge to mine parallel resources from the huge comparable data. In reality, it is very common that the parallel sentential and sub-sentential fragments do coexist, both of which are useful resources for SMT. Unfortunately, the previous work is usually restricted to either extract sentential or sub-sentential fragments, which will lead lots of useful resources unexploited. According to this consideration, this paper proposes a two-level parallel text extraction framework which can extract parallel sentences and parallel sub-sentential fragments simultaneously. The first step of the framework is to adopt a certain strategy to generate candidate sentences from huge comparable corpora and then mine parallel sentences and sub-sentential fragments from the candidates. Such an integration framework can make the utmost of the comparable corpora and help to alleviate the lack of parallel data. 2.Under the framework, the strategies for candidate sentence generation are studied. Mining parallel text from comparable data directly will confront vast search space and huge computations which will make the process to be too slow to be applied in the pratical application. In order to solve the problem, the paper employs two strategies for candidate sentence generation to locate ...
修改评论