Face Forgery Detection by 3D Decomposition and Composition Search

doi:10.1109/TPAMI.2022.3233586

	Face Forgery Detection by 3D Decomposition and Composition Search
	Zhu, Xiangyu1,2,3 ; Fei, Hongyan 1,2,3; Zhang, Bin 4; Zhang, Tianshuo 1,2,3; Zhang, Xiaoyu 4; Li, Stan Z.5 ; Lei, Zhen1,2,3,6
发表期刊	IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
ISSN	0162-8828
	2023-07-01
卷号	45 期号:7 页码:8342-8357
通讯作者	Lei, Zhen(zlei@nlpr.ia.ac.cn)
摘要	Detecting digital face manipulation has attracted extensive attention due to fake media's potential risks to the public. However, recent advances have been able to reduce the forgery signals to a low magnitude. Decomposition, which reversibly decomposes an image into several constituent elements, is a promising way to highlight the hidden forgery details. In this paper, we investigate a novel 3D decomposition based method that considers a face image as the production of the interaction between 3D geometry and lighting environment. Specifically, we disentangle a face image into four graphics components including 3D shape, lighting, common texture, and identity texture, which are respectively constrained by 3D morphable model, harmonic reflectance illumination, and PCA texture model. Meanwhile, we build a fine-grained morphing network to predict 3D shapes with pixel-level accuracy to reduce the noise in the decomposed elements. Moreover, we propose a composition search strategy that enables an automatic construction of an architecture to mine forgery clues from forgery-relevant components. Extensive experiments validate that the decomposed components highlight forgery artifacts, and the searched architecture extracts discriminative forgery features. Thus, our method achieves the state-of-the-art performance.
关键词	Faces Forgery Three-dimensional displays Face recognition Feature extraction Lighting Computer architecture Composition search differentiable search fake face forgery detection 3D decomposition 3D face model
DOI	10.1109/TPAMI.2022.3233586
关键词[WOS]	IMAGE
收录类别	SCI
语种	英语
资助项目	National Key Research & Development Program[2020AAA0140000] ; Chinese National Natural Science Foundation[62176256] ; Chinese National Natural Science Foundation[62276254] ; Chinese National Natural Science Foundation[62106264] ; Youth Innovation Promotion Association CAS[Y2021131] ; InnoHK program
项目资助者	National Key Research & Development Program ; Chinese National Natural Science Foundation ; Youth Innovation Promotion Association CAS ; InnoHK program
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:001004665900026
出版者	IEEE COMPUTER SOC
引用统计	被引频次：5[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/53661
专题	多模态人工智能系统全国重点实验室
通讯作者	Lei, Zhen
作者单位	1.Chinese Acad Sci, Ctr Biometr & Secur Res, Inst Automat, Beijing 100190, Peoples R China 2.Chinese Acad Sci, Natl Lab Pattern Recognit, Inst Automat, Beijing 100190, Peoples R China 3.UCAS, Sch Artificial Intelligence, Beijing 100049, Peoples R China 4.Chinese Acad Sci, Inst Informat Engn, Beijing 100045, Peoples R China 5.Westlake Univ, Sch Engn, Hangzhou 310024, Peoples R China 6.Chinese Acad Sci, Ctr Artificial Intelligence & Robot, Hong Kong Inst Sci & Innovat, Hong Kong, Peoples R China
第一作者单位	模式识别国家重点实验室
通讯作者单位	模式识别国家重点实验室
推荐引用方式 GB/T 7714	Zhu, Xiangyu,Fei, Hongyan,Zhang, Bin,et al. Face Forgery Detection by 3D Decomposition and Composition Search[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2023,45(7):8342-8357.
APA	Zhu, Xiangyu.,Fei, Hongyan.,Zhang, Bin.,Zhang, Tianshuo.,Zhang, Xiaoyu.,...&Lei, Zhen.(2023).Face Forgery Detection by 3D Decomposition and Composition Search.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,45(7),8342-8357.
MLA	Zhu, Xiangyu,et al."Face Forgery Detection by 3D Decomposition and Composition Search".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 45.7(2023):8342-8357.