毕业生知识库

随着移动互联网的持续发展，数据量呈爆炸式增长，知识图谱成为有效管理
这些海量数据的工具。在知识图谱中，实体和概念被表示为节点，而这些节点之
间的联系则通过边的形式明确呈现。大规模知识图谱的一个显著特性是其数据
呈长尾分布，因此如何准确地预测知识图谱中的低频关系是当前研究中亟待解
决的问题。本文通过结合深度学习、预训练语言模型和大语言模型中的先进技
术，研究基于交互学习、文本增强和大模型精排的知识图谱小样本学习方法。本
论文的主要工作内容和研究贡献总结如下：
1. 基于交互学习的小样本学习方法
针对现有方法中支持集和查询集在建模过程中互相独立的问题，本文提出
了基于交互学习的小样本学习方法。现有研究方法中，由于分别对支持集和查询
集建模，导致不同关系任务的查询集三元组向量相同，且由于向量维度限制，难
以实现关系特征的多角度和多层次表示。此外，查询的头实体和尾实体间可能存
在多种关系，其中不相关的关系可能干扰任务关系的语义表示。为了更好适应
各类关系任务，模型需要能根据不同关系类型建立动态的向量表示。这种任务
关系语义信息隐含在支持集内，本文通过事实交互学习方式来发掘此隐含信息，
并将其用于调整查询集的建模。为了增强模型泛化能力，在模型中加入了自适应
损失模块，使模型对难以分类的负样本更为关注，提升了该模型的鲁棒性。经过
在两个公开基准数据集上进行实验，验证了方法的有效性。
2. 实体描述增强的小样本学习方法
为了解决由于背景知识图谱信息不足而导致实体表示错误的问题，可以考
虑采用多种方法来增强实体表达。在本文中采用了为实体提供相关文本描述的
方法，以便更全面地描绘出实体的属性和关系。本文提出了异构信息融合的方法，
通过单层图神经网络提取实体在背景知识图谱上的邻域信息，同时使用BERT 提
取实体的文本语义信息。为了有效区分不同信息特征的重要性，本文设计了一个
门控网络，用于在特征融合过程中进行自适应权重调控，从而更好地利用不同来
源的信息。为了验证模型的效果，本文提出了FB15K-237-One 数据集。在该数
据集上，本文的模型取得了最优性能。通过本文的研究工作，成功解决了因背
景知识图谱信息不足导致的实体表示错误问题。
3. 基于大模型精排的小样本学习方法
在知识图谱小样本学习领域，大模型仍处于探索阶段，本文提出了一种基于
大模型精排的小样本学习方法。随着自然语言处理技术的不断演变，预训练语言
模型已逐步发展为大语言模型，其卓越的语义理解和推理能力在各领域均展现
出优异性能。在实体描述增强的小样本学习方法的研究基础上，本文通过与高
效微调方式结合，实现了在消费级中低端设备上微调大模型进行知识图谱小样
本学习。具体实现方式为，首先利用召回模型在FB15K-237-One 数据集上运行，得到重排数据集，然后将其构建为易于LLaMA2 理解的形式，最后结合高效微调方法来微调参数。实验结果显示，基于LoRA 微调的LLaMa2 在重排数据集上表现出色，展现了其在知识图谱小样本学习任务上的有效性和优越性，为小样本知识图谱补全提供了一种新的思路和途径。

With the continuous development of the internet, the amount of data is experiencing explosive growth, and knowledge graph has become an effective tool for managing these massive data. In a knowledge graph, entities and concepts are represented as nodes, and the connections between these nodes are explicitly presented in the form of edges. A notable characteristic of large-scale knowledge graphs is that their data follows a long-tail distribution, so accurately predicting low-frequency relationships in the knowledge graph is a pressing issue in current research. This paper combines advanced technologies from deep learning, pre-trained language models, and large language models to study a knowledge graph few-shot learning method based on interactive learning, text augmentation, and large model precise ranking.
The main work content and research contributions of this paper are summarized as follows:

1. Few-shot learning method based on interactive learning

Regarding the issue of the independence of the support set and query set in existing methods during the modeling process, this paper proposes a few-shot learning method based on interactive learning. In current research methods, modeling the support set and query set separately results in the same triplet vector for different relation tasks in the query set, and due to limitations in vector dimensions, it is difficult to achieve multi-angle and multi-level representations of relation features. In addition, there may be multiple relationships between the head and tail entities of a query, and unrelated relationships may interfere with the semantic representation of task relationships. In order to better adapt to various relationship tasks, the model needs to establish dynamic vector representations based on different types of relationships. This semantic information about task relationships is implicitly contained in the support set. This paper explores this implicit information through interactive learning of facts and uses it to adjust the modeling of the query set. To enhance the model's generalization capability, an adaptive loss module is added to the model, which allows the model to pay more attention to difficult-to-classify negative samples, thereby improving the robustness of the model. Experimental validation on two public benchmark datasets confirms the effectiveness of the method.

2. Entity Description-enhanced few-shot Learning Method

In order to address the issue of entity representation errors caused by insufficient background knowledge graph information, various methods can be considered to enhance entity expressions. In this paper, a method of providing relevant text descriptions for entities is employed to more comprehensively depict the attributes and relationships of entities. This paper proposes a method of heterogeneous information fusion, which extracts the neighborhood information of entities on the background knowledge graph using a single-layer graph neural network, while also using BERT to extract the textual semantic information of entities. To effectively differentiate the importance of different information features, this paper designs a gating network to adaptively regulate weights during the feature fusion process, thereby better utilizing information from different sources. To validate the model's effectiveness, this paper introduces the FB15K-237-One dataset. On this dataset, the model proposed in this study achieves optimal performance. Through the research presented in this paper, the issue of entity representation errors caused by insufficient background knowledge graph information has been successfully resolved.

3. Few-shot learning method based on large language model fine ranking

In the field of few-shot learning in knowledge graph, large models are still in the exploration stage. This article proposes a few-shot learning method based on large-model fine ranking. With the continuous evolution of natural language processing technology, pre-trained language models have gradually developed into large language models, demonstrating outstanding semantic understanding and reasoning capabilities in various domains. Building upon the research on few-shot learning methods with enhanced entity descriptions, this study integrates with an efficient fine-tuning approach to achieve fine-tuning of large models for few-shot learning in knowledge graph on consumer-grade mid-to-low-end devices. The specific implementation involves running a recall model on the FB15K-237-One dataset to obtain the rearranged dataset, which is then transformed into a format that is easy for LLaMa2 to understand, and finally fine-tuning the parameters using efficient fine-tuning methods. Experimental results show that LLaMa2 fine-tuned based on LoRA performs well on the rearranged dataset, demonstrating its effectiveness and superiority in the task of few-shot learning in knowledge graph, providing a new approach and avenue for completing few-shot knowledge graphs.