CASIA OpenIR  > 毕业生  > 博士学位论文
面向知识库问答的语义表示与计算关键技术研究
郝彦超
学位类型博士
导师赵军
2018-12-05
学位授予单位中国科学院自动化研究所
学位授予地点中国科学院自动化研究所
学科专业模式识别与智能系统
关键词自然语言处理
中文摘要

      随着互联网的不断发展,知识库作为知识的集合,数量在日益增长。为了对知识进行萃取,更好地服务用户,使用自然语言作为交互工具的知识库问答应运而生。作为自然语言处理和人工智能领域的重要任务,知识库问答受到了学术界和产业界的广泛关注。

       知识库问答从技术范式上可以分为两种:基于语义解析的问答范式和基于语义匹配的问答范式。前者是将用户提出的自然语言问句转化为基于符号逻辑的结构化查询语言,在知识库上查询得到答案。这种技术范式的优点是准确率高,可以处理复杂问句,但一个严重的缺点是强烈依赖辞典和人工设定的匹配规则。后者将知识库问答任务转化为语义匹配问题,首先对自然语言问句和知识库中资源的语义合理表示,然后根据两者的匹配程度给出答案。这种技术范式的优点是不需要人工设计特征和规则,相比来说,可扩展性强,更适合大规模知识库应用。

       面向大规模知识库问答,本文研究形式上更自由、可扩展性更强的基于语义匹配的知识库问答技术范式,其核心是语义的表示与相似度的计算。但是存在如下难点问题:1)如何表示问句的复杂语义,让系统深层理解问句;2)知识库自身具有不完备性,如何对其进行知识扩充,加强问答过程中的答案候选;3)自然语言表达具有自由性和丰富性,如何结合知识库进行它们的联合表示以及深层语义计算。面对知识库问答中的难点与挑战,本文针对语义表示与计算这个核心问题,分别从问句端、答案端(知识库端)、联合表示三个角度面向知识库问答任务进行了研究与探索,具体研究成果如下:

       一、基于模板改写的问句语义表示。在进行问句理解时,传统流水线系统存在错误级联累积问题,严重影响了知识库问答系统对问句的理解能力。本文针对这一关键问题,提出一个新的模板改写步骤来缓解问句理解过程中的错误累积问题,以更好地理解问句。该方法利用模板库中的高质量模板对当前问句的模板分析结果进行改写,然后根据问句模板得到主语实体指称项,将其与知识库资源进行对应后得到的候选主语实体以及与其连接的谓词关系当做候选事实集合。为了对候选的主语实体-谓词关系对进行排序,本文设计了一种联合事实选择策略,一方面利用关系检测来对候选主语实体进行重排序,另一方面采取嵌套式的结构表征问句中词的语义和问句的组合语义,同时利用多级别的编码和多维度的信息对整个过程进行增强表示,以更好地表征语义匹配过程。在公开数据集SimpleQuestions上的实验证明,本文提出的方法可以更好地表示问句语义,在性能上超过了目前最先进的方法,取得了显著的效果。

       二、基于实体对齐的知识库表示学习与融合。知识库的对齐融合对于知识库知识扩充,加强答案候选有着实际意义。当前知识库对齐融合方法大多是基于内容相似度的计算与传播,当面临跨语言或者不同编码的情形下,此类方法则不能很好地提供计算支撑。本文基于知识库表示学习经典工作TransE,提出一种基于种子实体对齐的知识库联合表示学习与融合方法,在一个统一的嵌入式向量空间中联合学习知识库的嵌入式表示,利用知识库的全局结构信息进行对齐与融合,进而增强答案候选,缓解知识库不完备性问题。同时本文基于真实世界大规模知识库构造了两个包含丰富关系和结构信息的数据集,分别为同源异构的FB15K和多源异构的DB-FB。在这两个数据集上,证明了本文提出方法的有效性。

       三、基于交互关注机制的语义表示与计算。现有表示学习和语义匹配方法通常关注于答案端资源的表示,对自然语言问句的表示一般仅仅通过循环神经网络或卷积神经网络进行固定表示。同时,将问句与知识库资源进行全匹配,并未有效地挑选有用的特征。本文提出一种基于交互关注机制的深度匹配模型,分别从答案到问句的关注以及问句到答案的关注两个方面入手,通过交互关注机制动态地表示自然语言问句和它们与相应知识库候选资源匹配的得分。本文提出的交互关注模型更符合直观上人们阅读问题搜索答案的重读机制。通过每一次对答案特征的重读,对问句语义进行动态调整与表示。当读完所有答案特征并且得到所有的得分之后,最后问句与答案之间的相似度得分是一个对上述所有得分的加权求和。本文认为这种动态的机制有助于系统理解问句,使之更好地与知识库资源进行深度匹配。同时,本文引入了背景知识库的全局知识来对知识库资源进行泛化,使之更好地与自然语言问句进行语义匹配计算。在公开数据集WebQuestions上的实验证明了本文提出的方法可以取得更好的结果。

英文摘要

     With the development of the Internet, the amount of knowledge bases (KBs) has been increasing for years. In order to extract, organize knowledge and serve users better, question answering over knowledge bases (KB-QA) which takes natural language as interface comes into being. As a significant task of natural language processing and artificial intelligence, KB-QA has attracted the attention from both academia and industry.

      To study the KB-QA task more comprehensively, this dissertation classifies the technological paradigm of the problem from two different views, i.e., semantic parsing based technique and semantic matching based technique. The former technique is to transform the users' natural language queries into symbolic logic based structured query language, and the answer could be obtained by running the structured query language over KB. The advantages of this technique are that it is highly accurate in precision, and has the ability of handling complex natural language sentences. But a serious fault is that it heavily depends on lexicons and manually designed rules, which leads to the difficulty of expanding to large scale KB applications. The latter technique converts the KB-QA task into semantic similarity computation and matching problem. The answers are given according to the matching degree between the semantic of natural language sentences and candidate answers in KB. This kind of technique is free from manually designed features or rules, and more capable of being applied to large scale KB.

      Aiming at large scale KB-QA, this dissertation studies the semantic matching based technique which has more free expressions and extensible framework. The core of this technique is semantic representation and similarity computation. While we have following challenges: 1) How to represent the complex semantic of questions in order to let QA systems understand questions deeply? 2) Generally a single KB is incomplete. How to expand its knowledge volume in order to enhance the candidate answers in QA procedure? 3) Natural language is free and expressive. How to understand questions according to KB resources and calculate the semantic of questions and KB resources jointly? Based on the difficulties and challenges in KB-QA, this dissertation solves the semantic representation and computation problem in KB-QA from three aspects: sentence end, answer end (KB end) and joint representation. The main contents and achievements are as follows:

      1) Pattern-revising Enhanced Sentence Semantic Representation. There is an error propagation problem in traditional pipeline KB-QA systems when trying to understand natural language sentences. This problem severely affects KB-QA systems in understanding natural language sentences. This dissertation addresses the problem by putting forward a novel sentence pattern revising procedure to alleviate the error propagation problem in order to let KB-QA systems understand questions better. The pattern revising procedure tries to find a question pattern which is more suitable for current question. Then we can get candidate subjects according to the corresponding revised question pattern. In order to learn to rank candidate subject-predicate pairs to enable the relevant facts retrieval given a question, we propose to do joint fact selection enhanced by relation detection. We adopt a nested structure to represent the semantic of words and the compositional semantic of sentences. Multi-level encodings and multi-dimension information are leveraged to strengthen the whole procedure. The experimental results on SimpleQuestions dataset demonstrate the effectiveness of the proposed approach, outperforming the current state-of-the-art by an absolute large margin.

      2) Entity Alignments based Knowledge Bases Representation and Fusion. The fusion of KBs is meaningful for knowledge expansion, and helpful to enhance answer candidates in practice. Instead of using content similarity and propagation based methods, we think the structure information of KBs is also important for KB alignment. When faced with the cross-linguistic or different encoding situation, what we can leverage is only the structure information of KBs. Based on TransE, in order to fuse KBs, we propose a model which jointly learns the embeddings of multiple KBs in a uniform vector space to align entities in KBs, based on the structure information. Our aim is to enhance the representation of answer end in KB-QA suitably, and alleviate the problem of incompleteness in KBs. To utilize structure information of KBs, we construct two datasets which have abundant relationships and rich structure information, including FB15K dataset and DB-FB dataset based on real-world large scale KB.  The results on the two datasets show that the proposed approach which only utilize the structure information of KBs is effective.

      3) Cross-attention based Semantic Representation and Computation for Question Answering over Knowledge Base. Previous work put more emphasis on the representation of candidate answers' features, and the question is converted into a fixed vector by recurrent neural networks or convolutional neural networks regardless of its candidate answers. At the same time, they use all the features of the KB resources to match the question regardless of the differences between the features. This simple representation strategy is not easy to express the proper information conveyed in the question according to different candidate answers. Hence, we present an end-to-end neural network model to represent the questions and their corresponding scores dynamically according to the various features of candidate answers via cross-attention mechanism, including the answer-towards-question attention part and the question-towards-answer attention part. The proposed cross-attention model could also be intuitively interpreted as a re-reading mechanism. We represent the semantic of sentences dynamically according to the re-reading every time. When we finish reading all the aspects of candidate answers and getting all the matching scores of corresponding features, the final similarity score is a weighted sum of all the matching scores. We believe this mechanism is beneficial for the systems to understand the question better with the help of corresponding answers' features, and it could lead to a better deep matching calculation between the semantic of sentence and resources in knowledge base. In addition, we leverage the global knowledge inside the underlying KB, aiming at integrating the rich KB information into the representation of the answers' features. As a result, it could alleviate the out-of-vocabulary (OOV) problem, which helps the cross-attention model represent the question more precisely. The experimental results on WebQuestions dataset demonstrate the effectiveness of the proposed approach.

页数138
语种中文
文献类型学位论文
条目标识符http://ir.ia.ac.cn/handle/173211/23097
专题毕业生_博士学位论文
推荐引用方式
GB/T 7714
郝彦超. 面向知识库问答的语义表示与计算关键技术研究[D]. 中国科学院自动化研究所. 中国科学院自动化研究所,2018.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
Thesis.pdf(6457KB)学位论文 暂不开放CC BY-NC-SA请求全文
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[郝彦超]的文章
百度学术
百度学术中相似的文章
[郝彦超]的文章
必应学术
必应学术中相似的文章
[郝彦超]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。