模式识别实验室知识库

人脸图像作为最普遍的图像种类之一，广泛存在于现实生活之中，这无疑为
研究人员基于此类数据进行算法研究和应用开发提供了便利。除了海量数据的
特点，人脸还具有多种属性，包括：身份，年龄，表情，姿态和化妆等，人脸相
关的问题一直以来得到科研工作者的青睐，尤其在大数据驱动的深度学习技术
快速发展下，相关研究都取得了重大突破，其中一些相对成熟的技术，如人脸识
别和人脸合成，改变我们生活的同时也让大众感受到了人工智能的魅力。然而，
由于人的面部运动、环境或时间因素导致的纹理和细节变化是不可避免的，合成
或编辑带有这些属性的图像得到期望的、高质量的和接近真实数据的图像仍是
一个巨大的挑战。
 

人脸属性分析可以简单地分为属性识别和属性编辑任务，其中属性编辑任
务的开展也会辅助属性识别发展，本课题将主要对属性编辑展开研究，尤其是人
脸表情属性的编辑研究。本文将以生成对抗网络为基础，探索不同网络结构在多
任务上的性能，设计具有鲁棒性的模型。具体研究内容如下：
 

• 提出了一种利用人脸关键点信息指导基于生成对抗网络的人脸表情编辑
方法，通过融合两个子生成对抗网络在一个框架中，可以同时完成表情合成和去
除任务。这样的操作会带来三个好处：第一点是可以稳定训练，提升生成的表情
人脸图像效果；第二点是利用提出方法结合传统方法进行人脸表情交换任务；第
三点是利用表情去除分支可以消除表情变化所带来的人脸识别性能的下降。
 

• 提出了一种基于生成对抗网络的人脸解析预测和表情编辑方法，通过设
计控制信息指导人脸解析预测过程，完成此阶段的训练后，利用人脸解析预测结
果辅助人脸编辑过程。其中，控制信息的加入可以更方便地控制人脸解析结果，
得到不同表情连续变化的解析预测输出。而富含人脸语义和结构信息的人脸解
析结果作为辅助信息可以帮助改善人脸表情合成效果。
 

Facial images as one of the most common types of images are available in our daily life, so there is no doubt that it is convenient for researchers to do some algorithms design and applications developments based on such images. Apart from that, face contains a variety of attributes, including identity, age, facial expression and makeup, etc. The problems related to face have drawn much attention from both academic community and industries. In particular, with the rapid development of deep learning technology driven by big data, many tasks in various fields have been advanced a lot. Some comparatively mature techniques, such as face recognition and face image synthesis, have changed our life tremendously in a positive way, making the public have a sense of satisfaction of artificial intelligence. However, owing to unavoidable texture and details variations caused by facial movements, environment and time, it is challengeable to synthesize and edit facial images with specific attributes to obtain high-quality and approximately realistic results.

To make it simple, facial attribute analysis can be divided into two parts: attribute recognition, and attribute editing which can promote the progress of attribute recognition. This project will focus on attribute editing, especially facial expression editing. More concretely, based on generative adversarial network, this work will explore the performances of distinct network structures, with the purpose of designing more generalized models to solve expression editing. The contributions of this thesis are summarized as follows:

• This thesis proposes a generative adversarial network based method editing facial expressions, guided by facial landmarks. Specifically, two sub generative adversarial networks are fused into one framework to simultaneously address two opposite tasks: expression synthesis and expression removal, which brings three benefits: 1) stabilizing the process of training and improving the quality of edited facial images; 2) combining the traditional method to transfer expressions between two people; 3) keeping the performance of face recognition using the expression removal branch when facing expression variations.
 

• An architecture based on generative adversarial theory is introduced to cope with face parsing prediction and facial expression editing. This work utilizes controllable variations to control the procedure of face parsing prediction. After that, the predicted results can assist facial expression editing. It is true that users can conveniently control features of expected results, both the kinds of expressions and degrees of expressions,
with such controllable information. In addition, due to the sufficiently semantic and structural information of face parsing results, it is evident that this information can enhance the final outputs.