|Place of Conferral||中国科学院自动化研究所|
With the development of the Internet, the amount of knowledge bases (KBs) has been increasing for years. In order to extract, organize knowledge and serve users better, question answering over knowledge bases (KB-QA) which takes natural language as interface comes into being. As a significant task of natural language processing and artificial intelligence, KB-QA has attracted the attention from both academia and industry.
To study the KB-QA task more comprehensively, this dissertation classifies the technological paradigm of the problem from two different views, i.e., semantic parsing based technique and semantic matching based technique. The former technique is to transform the users' natural language queries into symbolic logic based structured query language, and the answer could be obtained by running the structured query language over KB. The advantages of this technique are that it is highly accurate in precision, and has the ability of handling complex natural language sentences. But a serious fault is that it heavily depends on lexicons and manually designed rules, which leads to the difficulty of expanding to large scale KB applications. The latter technique converts the KB-QA task into semantic similarity computation and matching problem. The answers are given according to the matching degree between the semantic of natural language sentences and candidate answers in KB. This kind of technique is free from manually designed features or rules, and more capable of being applied to large scale KB.
Aiming at large scale KB-QA, this dissertation studies the semantic matching based technique which has more free expressions and extensible framework. The core of this technique is semantic representation and similarity computation. While we have following challenges: 1) How to represent the complex semantic of questions in order to let QA systems understand questions deeply? 2) Generally a single KB is incomplete. How to expand its knowledge volume in order to enhance the candidate answers in QA procedure? 3) Natural language is free and expressive. How to understand questions according to KB resources and calculate the semantic of questions and KB resources jointly? Based on the difficulties and challenges in KB-QA, this dissertation solves the semantic representation and computation problem in KB-QA from three aspects: sentence end, answer end (KB end) and joint representation. The main contents and achievements are as follows:
1) Pattern-revising Enhanced Sentence Semantic Representation. There is an error propagation problem in traditional pipeline KB-QA systems when trying to understand natural language sentences. This problem severely affects KB-QA systems in understanding natural language sentences. This dissertation addresses the problem by putting forward a novel sentence pattern revising procedure to alleviate the error propagation problem in order to let KB-QA systems understand questions better. The pattern revising procedure tries to find a question pattern which is more suitable for current question. Then we can get candidate subjects according to the corresponding revised question pattern. In order to learn to rank candidate subject-predicate pairs to enable the relevant facts retrieval given a question, we propose to do joint fact selection enhanced by relation detection. We adopt a nested structure to represent the semantic of words and the compositional semantic of sentences. Multi-level encodings and multi-dimension information are leveraged to strengthen the whole procedure. The experimental results on SimpleQuestions dataset demonstrate the effectiveness of the proposed approach, outperforming the current state-of-the-art by an absolute large margin.
2) Entity Alignments based Knowledge Bases Representation and Fusion. The fusion of KBs is meaningful for knowledge expansion, and helpful to enhance answer candidates in practice. Instead of using content similarity and propagation based methods, we think the structure information of KBs is also important for KB alignment. When faced with the cross-linguistic or different encoding situation, what we can leverage is only the structure information of KBs. Based on TransE, in order to fuse KBs, we propose a model which jointly learns the embeddings of multiple KBs in a uniform vector space to align entities in KBs, based on the structure information. Our aim is to enhance the representation of answer end in KB-QA suitably, and alleviate the problem of incompleteness in KBs. To utilize structure information of KBs, we construct two datasets which have abundant relationships and rich structure information, including FB15K dataset and DB-FB dataset based on real-world large scale KB. The results on the two datasets show that the proposed approach which only utilize the structure information of KBs is effective.
3) Cross-attention based Semantic Representation and Computation for Question Answering over Knowledge Base. Previous work put more emphasis on the representation of candidate answers' features, and the question is converted into a fixed vector by recurrent neural networks or convolutional neural networks regardless of its candidate answers. At the same time, they use all the features of the KB resources to match the question regardless of the differences between the features. This simple representation strategy is not easy to express the proper information conveyed in the question according to different candidate answers. Hence, we present an end-to-end neural network model to represent the questions and their corresponding scores dynamically according to the various features of candidate answers via cross-attention mechanism, including the answer-towards-question attention part and the question-towards-answer attention part. The proposed cross-attention model could also be intuitively interpreted as a re-reading mechanism. We represent the semantic of sentences dynamically according to the re-reading every time. When we finish reading all the aspects of candidate answers and getting all the matching scores of corresponding features, the final similarity score is a weighted sum of all the matching scores. We believe this mechanism is beneficial for the systems to understand the question better with the help of corresponding answers' features, and it could lead to a better deep matching calculation between the semantic of sentence and resources in knowledge base. In addition, we leverage the global knowledge inside the underlying KB, aiming at integrating the rich KB information into the representation of the answers' features. As a result, it could alleviate the out-of-vocabulary (OOV) problem, which helps the cross-attention model represent the question more precisely. The experimental results on WebQuestions dataset demonstrate the effectiveness of the proposed approach.
|郝彦超. 面向知识库问答的语义表示与计算关键技术研究[D]. 中国科学院自动化研究所. 中国科学院自动化研究所,2018.|
|Files in This Item:|
|Thesis.pdf（6457KB）||学位论文||暂不开放||CC BY-NC-SA||Application Full Text|
|Recommend this item|
|Export to Endnote|
|Similar articles in Google Scholar|
|Similar articles in Baidu academic|
|Similar articles in Bing Scholar|
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.