毕业生知识库

近年来，深度学习依靠海量数据作为训练支撑，在计算机视觉、自然语言 处理、语音识别等领域取得了巨大的成功。然而，海量数据收集可能引起潜在 的隐私风险：收集数据的服务商可能会永久保存这些数据，而被收集数据的用 户无法删除数据，也无法控制这些数据的使用。在此隐私保护的需求下，现有 的分布式学习系统往往要求用户进行数据隔绝，即互相不共享数据，仅通过共 享模型梯度来协同训练人工智能模型。这种分布式训练框架很好地避免了隐私 的直接泄露。 然而，在分布式学习系统中，不共享数据是否就能保障足够的隐私安全性？ 本文针对基于注意力机制的模型结构，深入分析了注意力模型的内在隐私漏洞， 提出了在注意力模型上进行隐私攻击和防御的新方法。本文的主要贡献如下： • 分析了注意力机制的内在隐私漏洞，并推导了针对注意力模块的闭式解 梯度泄露攻击，在满足特定条件的注意力网络中，能够仅基于模型梯度，通过 闭式解攻击算法，实现对输入的无损恢复，从而最大程度地攻击用户隐私。 • 对于更广泛的注意力模型结构，本文提出了一种基于优化的梯度泄露攻 击方法。通过对可学习位置编码模块的梯度进行额外监督，通过优化算法找到 近似于原始输入的一个解。实验结果表明，这种方式得到的攻击效果能显著超 越之前最好的方法。 • 针对注意力网络的隐私缺陷和上述的两种攻击手段，本文调研了几种不 同的隐私防御手段，其中，将可学习位置编码替换成固定位置编码的防御策略 能够在仅轻微影响模型性能的情况下破坏隐私攻击的成立条件，在保证模型精 度的情况下最大程度地保证了模型的隐私安全性。

In recent years, deep learning has achieved great success in the fields of computer vision, natural language processing, and speech recognition, relying on massive data as training support. However, the collection of massive user data can pose potential privacy risks: the machine learning service providers who collect the data may store the data permanently, while the users can neither delete or control the use of the collected data. With the need of privacy protection, current distributive learning systems require users to keep local data isolated, which means multiple users jointly train an artificial intelligence model only by sharing model gradients instead of sharing private data. Therefore, this kind of  distributive training framework avoids direct leakage of privacy.

However, does isolating data sufficiently mean that privacy is protected? Focusing at attention-based deep learning models, this thesis thoroughly analyzes the inherent privacy vulnerability of the attention-based models, and proposes new methods of privacy attack and defense on attention-based models. The main contribution of this thesis are as follows:

1.We analyze the inherent privacy vulnerability of the attention module. By theoretical derivation, we propose a closed-form gradient leakage attack on the attention module. In specific attention-based networks, the closed-form attack can perfectly recover the original input from the model gradients, which can be viewed as the most severe attack towards user privacy.

2.For more general attention-based models, we propose an optimization-based gradient leakage attack method. With additional supervision on the directional information of learnable position embedding's gradient, our optimization-based attack can find a solution in the input space close to the original input. The experimental results show that the attack effect obtained by this method can surpass the previous state-of-the-art method.

3.We further investigate several privacy defense strategies. Among several defense strategies, we found that replacing the learnable position embedding with fixed position embedding can break the condition of proposed privacy attacks with acceptable performance drop. In this way, the privacy is secured without loss of accuracy.