CASIA OpenIR  > 模式识别国家重点实验室  > 自然语言处理
Comparison Study on Critical Components in Composition Model for Phrase Representation
Wang, Shaonan1,2; Zong, Chengqing1,2,3
Source PublicationACM TRANSACTIONS ON ASIAN AND LOW-RESOURCE LANGUAGE INFORMATION PROCESSING
2017-04-01
Volume16Issue:3Pages:25
SubtypeArticle
AbstractPhrase representation, an important step in many NLP tasks, involves representing phrases as continuousvalued vectors. This article presents detailed comparisons concerning the effects of word vectors, training data, and the composition and objective function used in a composition model for phrase representation. Specifically, we first discuss how the augmented word representations affect the performance of the composition model. Then, we investigate whether different types of training data influence the performance of the composition model and, if so, how they influence it. Finally, we evaluate combinations of different composition and objective functions and discuss the factors related to composition model performance. All evaluations were conducted in both English and Chinese. Our main findings are as follows: (1) The Additive model with semantic enhanced word vectors performs comparably to the state-of-the-art model; (2) The Additive model which updates augmented word vectors and the Matrix model with semantic enhanced word vectors systematically outperforms the state-of-the-art model in bigram and multi-word phrase similarity task, respectively; (3) Representing the high frequency phrases by estimating their surrounding contexts is a good training objective for bigram phrase similarity tasks; and (4) The performance gain of composition model with semantic enhanced word vectors is due to the composition function and the greater weight attached to important words. Previous works focus on the composition function; however, our findings indicate that other components in the composition model (especially word representation) make a critical difference in phrase representation.
KeywordPhrase Representation Composition Model Retrofitting Word Paraphrasing Mean Square Error Max-margin
WOS HeadingsScience & Technology ; Technology
DOI10.1145/3010088
WOS KeywordPhrase representation ; composition model ; retrofitting ; word paraphrasing ; mean square error ; max-margin
Indexed BySCI
Language英语
Funding OrganizationNatural Science Foundation of China(61333018) ; Strategic Priority Research Program of the CAS(XDB02070007)
WOS Research AreaComputer Science
WOS SubjectComputer Science, Artificial Intelligence
WOS IDWOS:000399087800002
Citation statistics
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/15088
Collection模式识别国家重点实验室_自然语言处理
Affiliation1.National Laboratory of Pattern Recognition, Institute of Automation
2.University of Chinese Academy of Sciences
3.CAS Center for Excellence in Brain Science and Intelligence Technology
Recommended Citation
GB/T 7714
Wang, Shaonan,Zong, Chengqing. Comparison Study on Critical Components in Composition Model for Phrase Representation[J]. ACM TRANSACTIONS ON ASIAN AND LOW-RESOURCE LANGUAGE INFORMATION PROCESSING,2017,16(3):25.
APA Wang, Shaonan,&Zong, Chengqing.(2017).Comparison Study on Critical Components in Composition Model for Phrase Representation.ACM TRANSACTIONS ON ASIAN AND LOW-RESOURCE LANGUAGE INFORMATION PROCESSING,16(3),25.
MLA Wang, Shaonan,et al."Comparison Study on Critical Components in Composition Model for Phrase Representation".ACM TRANSACTIONS ON ASIAN AND LOW-RESOURCE LANGUAGE INFORMATION PROCESSING 16.3(2017):25.
Files in This Item: Download All
File Name/Size DocType Version Access License
wangshaonan TALLPcom(516KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Wang, Shaonan]'s Articles
[Zong, Chengqing]'s Articles
Baidu academic
Similar articles in Baidu academic
[Wang, Shaonan]'s Articles
[Zong, Chengqing]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Wang, Shaonan]'s Articles
[Zong, Chengqing]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: wangshaonan TALLPcomparison study .pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.