毕业生知识库

大数据时代广泛存在的数据孤岛问题对人工智能的应用与落地提出了新的挑战，为进一步发挥人工智能和大数据的潜力，联邦生态从系统化的角度定义了以联邦数据、联邦控制、联邦管理、联邦服务为核心的统一整体，有助于推动从数据、服务到智能的自动化转变，这一从上游数据到下游应用的标准化、体系化的解决方案也为不同领域的实际管理需求提供了理论层面的指导。其中，联邦学习将为其提供算法层面的技术支持。
作为联邦生态的核心环节，联邦控制能够将联邦管理的抽象任务转化为具体的执行规则，并对联邦节点进行调度控制，包括制定具体的节点选择策略与协作规则。然而，受真实场景中不同联邦节点的数据采集条件、设备情况等各方面差异的影响，各节点所拥有的本地数据通常是非独立同分布的，这使得联邦训练的最终效果难以达到预期。因此，本文面向非独立同分布场景展开联邦控制方法的研究，主要工作内容如下：
1. 面向联邦控制的非独立同分布场景构建及分析。针对联邦协作场景中广泛存在标签分布偏斜、数据量偏斜等非独立同分布情况，本文首先模拟真实场景对两个公开的图像分类数据集 MNIST 和 Fashion-MNIST 进行了划分，构建了拥有不平衡类别数与样本数的客户端环境，并在此基础上进一步分析了不同分布与不同偏斜程度对联邦学习模型带来的影响，为设计训练效率更高、适应性更强
的联邦控制方法提供了基础。
2. 基于训练趋势评估的自适应联邦控制方法。为降低联邦控制在调度客户端节点进行协作时产生的高昂通信成本，本文从参与方选择与全局模型更新两个步骤出发，优化作为联邦控制中心的服务器对客户端的节点选择策略与信息整合规则。针对参与方选择步骤，本文设计了基于 Mann-Kendall 趋势检验法的训练趋势评估算法，通过跟踪各客户端的本地训练效果，自适应地选择每个全局
轮次中参与训练的客户端。针对全局模型更新步骤，本文在模型聚合时加入了梯度相似性系数，作为各参与方在模型更新中的权重。实验表明，该方法能够高效地完成联邦训练任务，并提高了全局模型的准确率等各项指标。
3. 基于相似性度量的个性化联邦控制方法。为进一步满足不同节点的个性化需求，本文设计了一种低通信成本与存储成本的分簇个性化联邦控制方法，以优化联邦控制的协作规则。首先，服务器会为各个客户端维护多个全局簇模型，当客户端完成本地训练后，服务器就会及时调度相似性度量方法为其更新集群归属，过程中无需额外的数据传输，也无需在服务器中维护大量的中间信息。此
外，基于正则化的本地训练方法也使得客户端的本地迭代更加高效，进一步提升了簇模型的个性化效果。实验结果表明，该方法不仅能够达到分簇训练的预期效果，满足客户端的个性化需求，还不会产生额外的通信成本和过多的计算开销。

The widespread problem of data silos in the era of big data poses new challenges for applying and implementing artificial intelligence. Federated learning aims to establish a secure collaborative mode of large-scale distributed devices, providing a new way to break down data silos. In order to further develop the potential of artificial intelligence and big data, the federated ecology defines a unified whole centred on federated data, federated control, federated management, and federated service from a systematic perspective, which helps to promote automated transformation from data, service to intelligence. This standardized and systematic solution from upstream data to downstream applications also provides theoretical guidance for practical management needs in different fields.

As the core of the federated ecology, federated control can transform the abstract tasks from federated management into specific execution rules and control the scheduling of federated nodes, including formulating node selection strategies and collaboration rules. However, due to the differences in data collection conditions and equipment conditions of different federated nodes in real scenarios, the local data is usually not independent and identically distributed (IID), which will harm the final effect of the federated model. Therefore, this thesis studies the federated control methods on non-independent and identically distributed (Non-IID) scenarios to solve the above problems. The main works are presented as follows:

1. Construction and analysis of Non-IID scenarios for federated control. Aiming at the widespread Non-IID situations such as label distribution skew and data volume skew in federated collaboration scenarios, this thesis firstly simulates real scenarios to divide two public image classification datasets MNIST and Fashion-MNIST, and further constructs datasets with the imbalanced number of classes and samples. Subsequently, this thesis further analyzes the impact of different distributions and different skew degree on the federated learning model, which provides a basis for designing a federated control method with higher training efficiency and greater adaptability.

2. An adaptive federated control method based on training trend evaluation. To reduce the high communication cost of the federated control when scheduling clients for cooperation, this thesis optimizes the node selection strategy and information integration rules of the server, which play the role of the federated control centre, starting from the two steps of participant selection and global model update. For the participant selection step, a training trend evaluation method based on the Mann-Kendall trend test is designed. By tracking the local training effect of each client, the most suitable clients will be selected to participate in the current round of training. For the global model update step, this method further adds the gradient similarity coefficient as the weight of each participant. Experiments show that this method can complete the federated training task efficiently and improve the indicators of the global model, such as accuracy.

3. A personalized federated control method based on similarity measures. To further meet the individual needs of different nodes, this thesis designs a clustered personalized federated control method with low communication and storage costs, which optimizes the collaborative rules of federated control. First, the server will maintain multiple global cluster models for all clients. When the client completes local training, the server will promptly schedule the similarity measurement method to update the cluster it belongs to. In this process, neither additional data transmission nor much intermediate information needs to be maintained. In addition, the regularization-based local training method also makes the local iteration more efficient, which further improves the personalization performance of the cluster models. Experimental results show that this method can achieve the effect of federated clustering training as expected and meet the individual needs of every client without generating additional communication costs and excessive computational overhead.