毕业生知识库

图数据在现实世界中广泛存在，它可以自然地表示不同的对象及其之间的复杂关系。例如，在社交网络中，用户可以表示为节点，用户之间的好友关系可以表示为边。随着神经网络和深度学习技术的不断发展，近年来图神经网络在图数据分析中逐渐成为主流方法。虽然相关研究已经取得一些进展，但现有方法在利用图结构信息方面仍然存在较大提升空间。首先，对于节点级别的研究，由于图数据结构的千变万化，节点之间通常存在错综复杂的关系。为了学到节点之间的相似性和差异性，图神经网络需要拟合一个非常复杂的非线性函数，这对于图神经网络的非线性表达能力提出了挑战。其次，在图级别任务中，图数据通常包含丰富的结构信息和语义信息，如何自适应地提取图数据中的关键信息，提升图神经网络的判别能力，也是一大挑战。另外，在特定的标签分布下，图神经网络的性能通常会受到限制。如何更好地挖掘图的拓扑结构和标签分布之间的关系，也具有重要意义。本文以充分利用图数据的结构信息为切入点，深入挖掘结构信息和特定问题之间的内在联系。具体地，在节点级别、图级别和标签级别三种不同层次的图任务中，研究如何增强图神经网络的非线性表达能力、关键信息的提取能力、和特定标签分布下的判别能力。本文取得的研究成果主要包含以下四项：

1. 提出一种基于邻居聚合和交互的思路来解决已有图神经网络非线性表达能力不足的问题。本文首先总结了已有的图神经网络，指出它们仅通过非线性激活函数来建模节点之间的非线性关系。接着从理论上证明非线性激活函数的本质作用是建模邻居之间的交互，并且这种交互作用很弱。为了增强图神经网络的非线性表达能力，本文提出显式地建模邻居之间的交互信息，然后把邻居聚合的信息和邻居交互的信息整合起来。由于邻居交互会产生很多的交互项，为了减小算法复杂度，本文对网络中的邻居交互模块作出一定的假设简化，并设计了双路交互网络。顾名思义，双路交互网络包含两个支路，其中一个支路负责邻居聚合模块的学习，另一个支路负责邻居交互项参数的学习。对这两个支路得到的结果做进一步融合，即可得到非线性表达增强的图神经网络。通过这种方式，节点之间的非线性关系可以被网络学到。该方法与现有图神经网络的算法复杂度保持一致，并且可以用于用于节点级别的分类任务和链接预测任务。大量实验验证了这种方法的有效性。

2. 提出一种基于二阶全局注意力的方法，用于图级别关键信息学习。图数据包含复杂多样的结构信息和语义信息，这些信息对图级别判别任务的影响可能不同，因此神经网络需要自适应地提取重要信息。针对此问题，本文从两方面设计全局注意力机制：首先，在通道方面，设计通道注意力机制对图表达的不同维度进行加权，从而对某些语义进行增强。特别地，考虑到不同通道语义之间存在关联关系，本文进一步提出一种二阶注意力机制。其次，由于同一个图中不同节点的重要性也不尽相同，本文还设计了节点注意力，对不同节点的权重进行自适应调节。实验结果表明，通道注意力和节点注意力可以有效学习图数据中的关键信息，最终提升图分类和图回归任务的性能。

3. 提出一种基于结构信息编码的多专家融合图神经网络。在标签分布不均匀的场景下，图神经网络的预测会偏向于多类样本，导致模型在少类样本上的表现很差。本文考虑到图的结构特征和标签分布不均匀之间的关系，提出了一种基于结构信息编码的多视图多专家融合策略：首先，在邻居聚合和图池化的过程中，分别设计了多样化消息传递机制和多样化消息读出机制。这两种机制可以提取不同的结构信息，并得到不同视图的表达。其中，每个视图下的表达对应一组多专家分类器。这样既可以充分考虑到图结构信息的影响，又可以保证多专家网络的多样性。在标签分布不均匀的图分类数据集上，大量的实验表明该方法可以提升图神经网络的整体性能，并显著提高少类样本的分类准确率。

4. 提出一种基于结构引导的多任务学习图神经网络。本文以经典的多任务分子预测场景为例，旨在解决训练集中的标签缺失问题。本文设计了一个分子-任务二部图，将训练集中缺失标签补全问题转化为该二部图上的边预测问题。该二部图的结构信息可以显式地引导模型学习分子和任务之间的复杂关联，包括分子之间的相似性、任务之间的相关性、以及分子和任务的共现关系等。在该二部图上进一步设计图神经网络并训练，可以生成伪标签，从而对缺失标签进行补全。此外，本文还提出了一种基于不确定度的伪标签选取策略，它可以提供高质量的伪标签，减少噪音的影响。实验表明，该方法不仅可以在缺失标签补全方面保证较高准确率，而且可以提升模型对于多任务学习的整体性能。

Graphs are ubiquitous in the real world, which can naturally represent different objects and their complex relationships. For example, in social networks, users can be represented as nodes and friendships between users can be represented as edges. With the continuous development of neural networks and deep learning, graph neural networks have gradually become the mainstream method for graph analytical tasks in recent years. Although some progress has been made in related research, there is still considerable room for improvement, particularly in making full use of graph structure information. Firstly, for node-level research, due to the diverse structure of the graph, there are often complex relationships between nodes. In order to judge the similarities and differences between nodes, graph neural networks need to learn an extremely complex non-linear mapping function, which poses a challenge to the non-linear expressive ability of graph neural networks. Secondly, in graph-level tasks, graphs typically contain rich structural and semantic information. How to adaptively extract key information and improve the discriminative ability of graph neural networks is also a major challenge. In addition, under a specific label distribution, the performance of graph neural networks is usually limited. How to better mine the relationships between the graph structure and the label distribution is also of great significance. Starting from fully utilizing the structure information of graphs, this thesis aims to explore the intrinsic relationships between structure information and specific problems. Specifically, from three different levels of graph analytical tasks, including node-level, graph-level, and label-level, investigation is conducted on how to enhance the non-linear expressive ability, the key information extraction ability, and the discriminative ability under specific label distribution. The main contributions of this thesis are summarized as follows:

1.This thesis proposes a method based on neighborhood interaction to tackle the problem of insufficient non-linear expressive ability of existing graph neural networks. We first summarize existing graph neural networks and point out that the only way to model non-linear relationships between nodes is the non-linear activation function. Then from the theoretical perspective,  we prove that the non-linear activation functions brings in the interaction between neighbors, albeit in a subtle way. In order to enhance the non-linear expressive ability of graph neural networks, we propose to explicitly model the interaction information between neighbors, and then the neighborhood aggregation and neighborhood interaction are integrated. Due to the fact that 
there are many interaction items in the neighborhood interaction, we make several assumptions to simplify the neighborhood interaction module in the network. Correspondingly, we design a two-way interaction network. As the name implies, a two-way interaction network consists of two branches, one of which is responsible for learning the neighborhood aggregation, and the other branch is responsible for learning the interaction parameters. The results obtained from these two branches are further fused. In this way, the non-linear relationship between neighbors can be learned by the network, and can boost the node classification task and the link prediction task. In particular, the complexity of the algorithm is consistent with existing methods. A large number of experiments have verified the effectiveness of this method.

2. This thesis proposes a method based on a second-order global attention mechanism for the learning of graph-level key information. Graphs contain complex and rich structural and semantic information, which may have different impacts on the discrimination task. Therefore, graph neural networks need to adaptively extract important information. To this end, this thesis designs the global attention mechanism from two perspectives: first, in the channel aspect, a channel attention mechanism is designed to weight different dimensions of the graph representation, thereby enhancing some semantics. In particular, considering the correlation between the semantics of different channels, we further propose a second-order attention mechanism. Secondly, considering different nodes contribute to the graph representation differently, we also design the node attention mechanism to adjust node weights adaptively. Experimental results show that  channel attention and node attention can effectively learn key information from graphs, leading to performance improvement in graph classification and graph regression tasks.

3. This thesis proposes a structure-encoding graph neural network based on mixture-of-experts. In scenarios where the distribution of labels is imbalanced, graph neural networks tend to predict towards the majority class, resulting in poor performance towards the minority class.  Considering the relationship between the  graph structure and the imbalanced distribution of labels, this thesis proposes a structure-encoding graph neural network based on mixture of multi-view experts. Firstly, in the process of neighborhood aggregation and graph pooling, a diversified message passing mechanism and a diversified message readout mechanism are designed, respectively. These two mechanisms can extract different structural information and obtain representations of different views. Among them, the representations under each view correspond to a set of multi expert classifiers.
This can not only consider the impact of graph structure information, but also ensure the diversity of the experts. On imbalanced graph classification datasets, a large number of experiments have shown that this method can boost the performance of graph neural networks, and significantly improve the classification accuracy of minority class.

4. This thesis proposes a structure-guided multi-task learning graph  neural network. Taking the classic scenario of multi-task molecular property prediction as an example, this thesis focuses on the common label missing phenomenon.
This thesis designs a molecular-task bipartite graph, which transforms the missing label imputation problem into a link prediction problem on the bipartite graph. The structure information of the bipartite graph can explicitly guide the model to learn the complex association relationships between molecules and tasks. A graph neural network is further designed on this bipartite graph, which can learn the similarity between molecules, the correlation between tasks, and the co-occurrence relationship between molecules and tasks. Besides, it can make reasonable predictions for missing labels. In addition, this thesis also proposes an uncertainty-aware pseudo-label selection strategy, which can provide reliable pseudo-labels and reduce the impact of noise. Experiments have shown that this method can provide more high-quality supervision information for graph neural networks, as well as improving the overall performance of multi-task graph neural networks.