CASIA OpenIR  > 毕业生  > 硕士学位论文
提高信息检索准确性的技术研究
Alternative TitleResearch on the Technologies for Improving the Precision of Information Retrieval
吕碧波
Subtype工学硕士
Thesis Advisor赵军
2006-06-11
Degree Grantor中国科学院研究生院
Place of Conferral中国科学院自动化研究所
Degree Discipline模式识别与智能系统
Keyword信息检索 准确性 语言模型 查询扩展 自然语言处理 Information Retrieval Precision Language Model Query Expansion Natural Language Processing
Abstract如何提高信息检索系统的准确性一直是研究人员最关心的问题。本文首先简单介绍了信息检索的一些基本概念及其评测的相关内容,然后从以下几个方面展开,研究提高信息检索系统准确性的一些技术: 1.比较信息检索的布尔模型、向量空间模型、概率模型和语言模型。实验表明:语言模型能通过调节平滑参数使得系统的准确性超过其它模型;而且测试集越大,词语的分布就越均匀,那么对文档的最大似然估计的置信度越大,系统的准确性也就越高。 2.研究查询扩展对系统准确性的影响。文中提到了两种查询扩展的技术—基于词典的扩展和基于相关反馈的扩展。实验表明:在前一种方法中,扩展名词、形容词和副词均可以使系统的准确率得到提升,而扩展动词反而会降低系统的准确率;通过词义消歧对扩展词进行噪音过滤在本文实验中没有取得应有的效果;在后一种方法中,本文提出了基于语言模型的扩展词特征选择方算法,与已往算法相比,本文提出的算法能提高系统的准确性。但是,在初次查询结果不佳的情况下,这种方法反而使得系统的准确性降低。 3.探索自然语言处理技术在信息检索中的应用。介绍了五种词间关系模型,即二阶语言模型、基于词对的语言模型、基于窗口的语言模型、依存语言模型和基于概念的语言模型。实验表明,与“词袋子”模型相比,上述五种模型都可以在一定程度上提高检索的准确率,其中窗口为3的语言模型使系统获得最高的准确性。 4.通过初始化查询权重,模拟了文档模型和查询模型。实验结果表明:只有当测试集比较大的时候,文档模型才能近似看成查询模型。 5.将网页评级技术引入了基于语言模型的信息检索框架。实验结果表明:区分网页的质量会提高信息检索的准确性。 最后介绍了作者参加TREC2005 HARD任务与863信息检索评测的情况。
Other AbstractHow to make Information Retrieval (IR) system more precise is the key issue that all the researchers of related fields pay special attention to. Firstly, the thesis introduces the basic concept of IR and its evaluation, and then presents some methods to increase the precision of IR system as follows: 1.Gives a comparison of Bool Model (BM)、Vector Space Model (VSM)、Probability Model (PM) and Language Model (LM). According to our experiments, the precision of LM based IR system is higher than the others through adjusting smoothing parameters. And we can learn that: for a large test collection, the distribution of words is even, system performance can be improved through giving more weights to the Maximum Likelihood Estimation (MLE) of Document Model (DM). 2.Investigates the improvement of precision of IR through Query Expansion (QE). Two techniques are used, i.e. dictionary based QE and Relevance Feedback (RF) based QE. For the first one, expanding nouns、adjectives and adverbs respectively will increase the precision of IR system, but the precision will get worse after expanding verbs , Word Sense Disambiguation (WSD) can not eliminate the noises introduced from QE; For the latter one, a new term selection method is proposed and it is proven to be better than classic ones according to our experiments. However, the precision will decrease after RF if the initial retrieval result is not good enough. 3.Studies the effects of incorporating Natural Language Processing (NLP) technologies into IR system. Five term-relation-based models are introduced, i.e. Bigram Model、Word-Pair Model、Window-Based Model 、Dependence Model and Concept-Based Model. Compared with “bag of words” based model, all the five term-relation-based models will improve the performance of IR system . Among them, the Window-Based IR system with window size of 3 outperforms the others. 4.Simulates Query Model (QM) and Document Model (DM) by initializing the weights the query terms. The experiments show that DM can approach QM only in the cases that the test collection becomes much larger. 5.Introduces Page Level (PL) to the frame of LM based IR system. The experiments show that distinguishing page quality can improve the performance of IR system. Finally, the paper introduces the details about our participations in TREC2005 HARD track and 863 Information Retrieval Evaluation.
shelfnumXWLW1020
Other Identifier200328014604141
Language中文
Document Type学位论文
Identifierhttp://ir.ia.ac.cn/handle/173211/7391
Collection毕业生_硕士学位论文
Recommended Citation
GB/T 7714
吕碧波. 提高信息检索准确性的技术研究[D]. 中国科学院自动化研究所. 中国科学院研究生院,2006.
Files in This Item:
File Name/Size DocType Version Access License
CASIA_20032801460414(1614KB) 暂不开放CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[吕碧波]'s Articles
Baidu academic
Similar articles in Baidu academic
[吕碧波]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[吕碧波]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.