毕业生知识库

知识图谱（Knowledge Graph）是以三元组为基本语义单元，以有向标签图为数据结构，从知识本体和知识实例两个层次，对世界万物进行体系化、规范化描述，并支持高效知识推理和语义计算的大规模知识系统。知识图谱不仅是实现认知智能的底层支撑和重要手段，还在很多现实应用中发挥着不可替代的作用。但是依靠专家人工编撰来构建知识图谱不仅要花费巨大的人力和金钱成本，还存在着知识覆盖率低、数据稀疏、更新缓慢等问题。因此，本文研究知识图谱自动构建技术。

近年来，受益于深度学习的发展，知识图谱自动构建技术取得了长足的进步。但将现有的技术应用到真实场景时还面临着以下问题：（1）在真实应用场景中，信息越来越多地以多模态形式出现，而现有方法缺乏对多模态信息的利用；（2）在真实应用场景中，数据管理与隐私保护的要求日益严格，而现有方法的训练过程需要暴露大量数据；（3）在真实应用场景中，线上部署要求模型精简高效，而现有方法存在模块冗

余和显式误差累积问题。本论文针对以上问题展开研究，研究成果和创新点如下：

基于多模态信息的中文命名实体识别。针对真实应用场景中信息越来越多地以多模态形式出现，而现有的命名实体识别方法绝大部分都只利用文本信息推断命名实体标签这一矛盾，研究了基于多模态信息的中文命名实体识别技术。主要贡献包括：（1）提出了语音文本双模态命名实体识别任务，阐述了语音模态在中文命名实体识别任务中的重要作用；（2）构建了国际上第一个人工标注的语音文本双模态中文命名实体识别数据集；（3）提出了一种基于多任务学习的多模态中文命名实体识别方法，该方法利用掩码联结时序分类机制捕获模态之间的对齐关系，并利用多任务学习框架进行联合训练。实验结果表明：（1）利用语音模态可以有效提升现有命名实体识别方法的性能，特别是能够有效地减少实体边界识别的误差；（2）提出的方法能够有效捕获对齐关系，从而提升命名实体识别的性能。

基于隐私联邦学习的实体关系抽取。针对真实应用场景中数据管理与隐私保护的要求日益严格，而现有方法的训练过程需要暴露大量数据这一矛盾，研究了基于隐私联邦学习的实体关系抽取技术。主要贡献包括：（1）提出了联邦远程监督关系抽取任务，阐述了将联邦学习与远程监督学习结合的意义； （2）提出了噪音鲁棒联邦学习方法，该方法通过建立跨平台之间的协作来缓解联邦远程监督关系抽取中的标签噪音问题；（3）提出了基于集成蒸馏的联邦训练框架，该框架通过模型知识迁移实现联邦中心聚合，进而降低联邦学习中的通信开销。实验结果表明：（1）提出的噪音鲁棒联邦学习方法能够有效缓解联邦设定下的标签噪音问题；（2）提出的基于集成蒸馏的训练框架能够有效降低通信开销。 

基于序列到集合的知识图谱一体化构建。针对真实应用场景中线上部署要求模型精简高效，而现有方法存在模块冗余和显式误差累积这一矛盾，研究了基于序列到集合的知识图谱一体化构建技术。主要贡献包括：（1）提出将知识图谱一体化构建过程建模为集合生成任务，并阐述了该建模方法的合理性；（2）提出了基于非自回归解码器的集合生成模块，该模块能够同时生成所有集合元素从而缓解序列到序列方法中需建模事实顺序的问题；（3）设计了面向集合的二部匹配损失函数，该函数通过匈牙利算法自动计算集合与集合之间的对应关系，进而以可微分的形式实现序列到集合模型的高效训练。 实验结果表明：（1）提出的方法可以在模块精简的情况下，实现知识的准确抽取；（2）提出的方法可以高效地实现模型训练以及新样本推断。

The knowledge graph is a large-scale knowledge system that takes triples as the basic semantic unit and directed label graph as the data structure, uses knowledge ontology and knowledge instance to describe all things in the world, and supports efficient knowledge reasoning and semantic computing. The knowledge graph is the bottom support to realize cognitive and plays an irreplaceable role in many practical applications. However, relying on constructing the knowledge graph manually not only costs huge manpower and money, but also arises some serious problems, such as low coverage of knowledge, sparse data and slow update. Therefore, this dissertation focuses on the key technologies of automatic knowledge graph construction.

In recent years, benefiting from deep learning, the key technologies of knowledge graph construction have made great progress. However, when applying these existing technologies to real scenarios, it still faces the following problems: (1) The information appears more and more in the form of multi-modal, but the existing methods lack the use of multi-modal information. (2) The requirements for data management and privacy protection are becoming increasingly strict, but the training process of the existing methods needs to expose a large amount of data. (3) Online deployment requires the model to be concise and efficient, but the existing methods use a pipeline architecture for knowledge graph construction, which is inherently prone to error propagation between its components. To solve the above problems, this dissertation proposed:

Chinese named entity recognition based on multimodal information. Due to that more and more information appears in multimodal form in real application scenarios, I proposed the named entity recognition based on multimodal. The main contributions include: (1) The task of Chinese multimodal named entity recognition with textual and acoustic contents is proposed. (2) The first human annotated Chinese multimodal named entity recognition dataset is constructed. (3) A multimodal multitask method is proposed, which introduces a speech-to-text alignment auxiliary task. The experimental results show that: (1) Using acoustic modal can effectively improve the performance of existing named entity recognition methods, and can effectively reduce boundary errors. (2) The proposed method can effectively capture the monotonic alignment between the acoustic modality and the textual modality, and armed with this alignment, the named entity recognition methods can perform better. 

Relation extraction based on privacy federated learning. Due to the increasingly strict requirements for data management and privacy protection in real application scenarios, I proposed the relation extraction based on privacy federated learning. The main contributions are as follows: (1) The task of distantly supervised relation extraction in federated settings is proposed. (2) The noise-robust federated learning is proposed, which can suppress noise by coordinating multiple platforms. (3) The ensemble distillation based federated training framework is proposed, which reduces the communication overhead by modeling federated aggregation as knowledge transfer. The experimental results show that: (1) The proposed noise-robust federated learning can effectively suppress the label noise under federated settings. (2) The ensemble distillation based federated training framework can effectively reduce the communication overhead.

Knowledge graph construction based on sequence-to-set. Due to that online deployment requires the model to be concise and efficient, I proposed knowledge graph construction based on sequence-to-set. The main contributions are as follows:(1) The sequence-to-set method is proposed, and the reason why modeling knowledge graph construction as a set generation problem is explained. (2) The non-autoregressive set generator is proposed, which can generate the set in one shot. (3) A bipartite matching loss function is proposed, which can evaluate the difference between sets via the Hungarian algorithm. The experimental results show that: (1) The proposed method can yield state-of-the-art results on multiple benchmark datasets. (2) The proposed method is very efficient in training and inference.