AOT: Appearance Optimal Transport Based Identity Swapping for Forgery Detection | |
Zhu, Hao1,2; Fu, Chaoyou1,3![]() ![]() ![]() | |
2020 | |
会议名称 | Advances in Neural Information Processing Systems |
会议日期 | 2020.12.6 |
会议地点 | 线上 |
摘要 | Recent studies have shown that the performance of forgery detection can be improved with diverse and challenging Deepfakes datasets. However, due to the lack of Deepfakes datasets with large variance in appearance, which can be hardly produced by recent identity swapping methods, the detection algorithm may fail in this situation. In this work, we provide a new identity swapping algorithm with large differences in appearance for face forgery detection. The appearance gaps mainly arise from the large discrepancies in illuminations and skin colors that widely exist in real-world scenarios. However, due to the difficulties of modeling the complex appearance mapping, it is challenging to transfer fine-grained appearances adaptively while preserving identity traits. This paper formulates appearance mapping as an optimal transport problem and proposes an Appearance Optimal Transport model (AOT) to formulate it in both latent and pixel space. Specifically, a relighting generator is designed to simulate the optimal transport plan. It is solved via minimizing the Wasserstein distance of the learned features in the latent space, enabling better performance and less computation than conventional optimization. To further refine the solution of the optimal transport plan, we develop a segmentation game to minimize the Wasserstein distance in the pixel space. A discriminator is introduced to distinguish the fake parts from a mix of real and fake image patches. Extensive experiments reveal that the superiority of our method when compared with state-of-the-art methods and the ability of our generated data to improve the performance of face forgery detection. |
文献类型 | 会议论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/48648 |
专题 | 模式识别实验室 |
通讯作者 | He, Ran |
作者单位 | 1.NLPR & CEBSIT & CRIPAC, CASIA 2.Anhui University 3.University of Chinese Academy of Sciences 4.SenseTime Research |
第一作者单位 | 模式识别国家重点实验室 |
通讯作者单位 | 模式识别国家重点实验室 |
推荐引用方式 GB/T 7714 | Zhu, Hao,Fu, Chaoyou,Wu, Qianyi,et al. AOT: Appearance Optimal Transport Based Identity Swapping for Forgery Detection[C],2020. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
NeurIPS_AOT.pdf(5219KB) | 会议论文 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论