|
The upgrading of intelligent communication devices and the rapid development of deep learning have provided favorable conditions for the widespread popularity of facial content synthesis. The threshold for technical operations has been greatly reduced, and network users can easily obtain facial images and videos, and use various editing software or open source code to perform AI faceswapping and facial editing, causing a lot of passion about this in social media and public platforms. While this technology can be used for positive applications, such as artistic creation, education, human-computer interaction and etc. The negative impact of malicious abuse is much greater than the positive impact. On the one hand, the face forgery technology develops rapidly and it can produce ultra-realistic visual content about human faces, which break up a traditional concept of "seeing is believing". On the other hand, malicious contents about human face often meets the curiosity of the public and has strong abilities to shape and distort their consciousness. Currently, there have been many illegal and criminal cases related to facial forgery, which have not only caused severe damage to individual reputation and financial security, but have also posed great threats to the social stability, the internet sovereignty and political security of our country. Therefore, conducting research on face forgery detection is imperative.
Until now, researchers have proposed many effective methods for face forgery detection, which perform well on public datasets and the detection accuracy increases constantly. However, there still exist many problems when applying these methods to real-world scenarios. For example, high-accuracy face forgery models can not be deployed directly due to the limitation of computing resources, model trained on specific dataset is unable to deal with newly emerged forgery types, and etc. In this thesis, we pay attention to face forgery detection in real-world scenarios, and choose face forgery detection model quantization and lifelong learning of as the key parts of our research. The main contributions are listed as follows:
1) A novel model compression method is proposed, which utilizes a two-stage scheme to obtain light-weight face forgery detection models. At the first stage, the key parameters of a high-accuracy face forgery detection model are compressed through using post-training quantization, that is, the weights and activation values are all converted from high-width floating points to low-width integers, which effectively reduces the memory footprint and computing consumption. At the second stage, the calibration set is obtained and fed into the light-weight face forgery detection model to calibrate the activation range dynamically during the forward passing process. We does not use data sampled from original training set as the calibration data. Instead, we fully mine statistic information contained in each batch normalization layer of the pre-training model through knowledge distillation, and then use them to guide the generator for synthesizing a batch of data which has a similar distribution to original training data. By this way, the calibration process can be implemented in data-free environment. In addition, it is known that the activation value depends on the type of activation function. Therefore, we find the most quantization-friendly activation function, ReLU6 , through theoretical analysis for further improving the performance of light-weight model. The proposed method is tested on two classical fake face datasets. The results show that the proposed method can successfully compress a series of advanced face forgery detection models to light-weight models, which still maintain a high performance. At the same time, the memory consumption and computing resources are significantly reduced.
2) A lifelong learning method is proposed, which utilizes feature reconstruction and feature joint-learning to make face forgery models have the ability to cope with new forgery types in non-examplar scenario. This method makes full use of the characteristics of face forgery detection. On the one hand, according to the differences among different types of forged faces, the lifelong learning of face forgery detection model can be modeled as "domain incremental learning", which trains model on face forgery datasets sequentially. On the other hand, according to the similarity between different types of forged faces, the face forgery detection model is split into two parts: a fixed feature extractor and a continuously adjusted classifier. In each training stage, a certain amount of reconstructed features originted from old domain and features newly extracted from current domain are fed to the classifier for joint training, which can fight against the catastrophic forgetting of old knowledge and balance the performance in different domains. Extensive experiments were carried out on 6 representative fake face datasets. The results show that the proposed method can make a face forgery detection model be adaptive to the continuous new domain while still maintaining performance well in the old domain. Compared with joint training or branch integration, the performance of the model is basically maintained and the cost of training resources is greatly reduced.
|
修改评论