模式识别实验室知识库

自深度学习兴起以来，人脸识别在理论研究和实际应用方面都取得了令人满意的结果。然而，大部分的人脸识别研究都集中在可控条件下的可见光人脸，在真实无约束场景中仍然存在挑战。比如，在特定的应用场景下，待匹配的人脸图像来源于其他模态下的非可见光人脸图像，现有的人脸识别网络无法正确匹配，这就促使了异质人脸识别任务的产生。跨光谱人脸识别作为异质人脸识别的子领域，在安全方面有着重要的应用，是当前生物识别领域研究最广泛的课题之一。由于近红外和可见光人脸图像存在着巨大的域差异，跨光谱人脸识别具有相当大的挑战性。同时，由于近红外数据难以获得，现有的数据集中成对的人脸图像数量有限，存在着严重的过拟合和泛化性弱的问题。本文针对数据受限给跨光谱人脸识别带来的这些问题展开研究，取得的主要研究成果如下：

1. 针对数据受限带来的过拟合问题，本文提出了一种新的自增强方法：混合对抗样本和Logits 重放。具体来说，该方法首先利用识别网络本身生成对抗样本，然后将它们与干净样本以插值的方式混合，以进行数据扩充。与此同时，本文根据跨域问题扩展了对抗样本的定义，旨在减少域差异，提取域不变特征。为了有效地利用了在大规模可见光数据集上获得的判别性特征，该方法进一步提出了一种通过logits 重放保持特征多样性的损失函数，有效地改善了混合对抗样本方法无法获得的特征多样性。实验表明，该方法有效地缓解了过拟合问题，显著提高了跨光谱的识别性能。

2. 针对数据受限带来的泛化性弱的问题，本文从信息论的角度出发，提出了一种近红外到可见光的解纠缠跨光谱信息识别方法。具体来说，为了将信息瓶颈理论应用到特征提取过程中，该方法将图像特征分解为两个互补的部分: 身份相关特征和身份无关特征。这样，信息瓶颈的优化可以通过使用一个解码器来完成，从而获得泛化性强的人脸特征。此外，该方法还提出了一个特征融合模块，以促进两种特征表示之间的解纠缠能力，同时可以最大化不同域下的身份相关特征的互信息来减少域差异。实验表明，本文的方法能够有效地提取出泛化性强的人脸特征，并同时提高跨光谱人脸识别的性能。

总之，本文提供了数据受限下跨光谱人脸识别存在问题的解决思路，同时还总结跨光谱人脸识别的最新进展，包括相关的研究方法，识别模型和数据库，并进一步对存在的问题及未来的研究方向做出了讨论。

Since the rise of deep learning, face recognition has achieved satisfactory results in both theoretical research and practical application. However, existing face recognition networks are all focused on visible face under controlled conditions, and there are still challenges in real unconstrained scenes. For example, in specific application scenarios, the face image to be matched comes from non-visible face image in other modalities, and the existing face recognition network cannot match correctly, which promotes the emergence of heterogeneous face recognition (HFR) tasks. As a sub-domain of heterogeneous face recognition, cross-spectral face recognition has important applications in security and is one of the most widely studied topics in biometric recognition. Due to the large domain discrepancy between near-infrared and visible face images, crossspectral face recognition is quite challenging. At the same time, the number of paired face images in existing datasets is limited because of the difficulty of obtaining near-infrare data, there are serious problems of over-fitting and weak generalization. This thesis studies these problems caused by limited data for cross-spectral face recognition, and the main work is summarized as follows:

1. To solve the problem of over-fitting caused by limited data, we propose a new self-augmentation method: Mixed Adversarial example and Logits Replay (MAELR). Specifically, we first generate adversarial examples and then mix them with clean examples in interpolation for data augmentation. At the same time, we extend the definition of adversarial examples to cross-domain problems. Benefiting from this extension, we can reduce domain discrepancy to extract domain-invariant features. We further propose a diversity preserving loss via logits replay, which effectively uses the discriminative features obtained on the large-scale VIS dataset. In this way, we improve the feature diversity that can not be obtained from mixed adversarial examples methods. Extensive experiments demonstrate that our method alleviates the over-fitting problem, thus significantly improving the recognition performance of cross-spectral face recognition.

2. Aiming at the problem of weak generalization caused by limited data, a Disentangled Cross-spectrum Information Recognition (DCIR) method is proposed from the perspective of information theory. Specifically, in order to apply information bottleneck theory into the process of feature extraction, we disentangled the image feature into two complementary parts: identity-relevant feature and identity-irrelevant feature. In this way, we can optimize the information bottleneck items by just using one decoder to obtain well-generalized features. Besides, we propose a feature fusion module to facilitate feature representation disentangled, and simultaneously we can reduce domain discrepancy by maximizing the mutual information of features in different domains. Extensive experiments demonstrate that our method can effectively extract highly generalized face features and improve the performance of cross-spectral face recognition.

In conclusion, this thesis provides solutions to the existing problems of cross-spectral face recognition with limited data, and summarizes the related work of cross-spectral face recognition, including related research methods, recognition models and datasets. In addition, this thesis further discusses the existing problems and future research directions.