Forged facial video detection framework based on multi-region temporal relationship feature

doi:10.1063/5.0125032

	Forged facial video detection framework based on multi-region temporal relationship feature
	Fang, Xing 1; Hao, YanNi 2; Luo, Yin2,3 ; Xu, Nan2,3 ; Cao, Jia2,3
发表期刊	AIP ADVANCES
	2023-08-01
卷号	13 期号:8 页码:8
通讯作者	Fang, Xing(fangxing@tju.edu.cn)
摘要	Face generation and manipulation techniques based on deep learning have enabled the creation of sophisticated forged facial videos, which are indistinguishable by human eyes. However, the illegal use of deep fake technology will have a serious impact on social stability, personal reputation, and even national security. Therefore, the detection technology of fake facial videos is of great significance to protect national security and maintain social order. Although the existing video-based fake face video detection technology has achieved good detection performance on the public fake face video database, there are still the following problems: (1) the existing technology uses a 2D attention mechanism to obtain local region features from face images and lacks a 3D attention mechanism to obtain local area features from face videos; (2) after obtaining local area features, the existing technology is directly used to classify or only model the inter-regional relationship of images without modeling the temporal relationship between regions of the video. This paper proposes a fake facial video detection framework based on multi-region temporal relationship features, including designing a three-dimensional attention mechanism to extract local features of multiple regions of the face from the video. In order to model the time series relationship between different face areas, a time series graph convolution neural network is also introduced to extract the time series relationship features between multiple areas. In order to model the time sequence relationship between different face regions, the convolution neural network of a time sequence diagram is also introduced to extract the characteristics of the time sequence relationship between multiple regions. Through the change characteristics of the time sequence relationship between face regions, the timing inconsistency of the face video is detected so as to determine whether the face has been deeply forged. Through experiments on multiple datasets, the experimental results of the model accuracy test show that the method proposed in the present invention achieves the highest detection accuracy, and the accuracy in the FaceForensics++ (low definition) dataset is 18.19% higher than that of the benchmark method. The experimental results of the generalization ability test show that the method proposed in the present invention achieves the highest generalization performance, and the detection accuracy in the Celeb-DF dataset is 11.92% higher than that of the benchmark method.
DOI	10.1063/5.0125032
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Plan Science and Technology Winter Olympics Key Project: Research and Application Demonstration of the Global Communication Service Platform for the Winter Olympics[2020YFF0305300]
项目资助者	National Key Research and Development Plan Science and Technology Winter Olympics Key Project: Research and Application Demonstration of the Global Communication Service Platform for the Winter Olympics
WOS研究方向	Science & Technology - Other Topics ; Materials Science ; Physics
WOS类目	Nanoscience & Nanotechnology ; Materials Science, Multidisciplinary ; Physics, Applied
WOS记录号	WOS:001052966400001
出版者	AIP Publishing
引用统计
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/54090
专题	舆论大数据科学与技术应用联合实验室
通讯作者	Fang, Xing
作者单位	1.Tianjin Univ, Sch Management, Tianjin 300072, Peoples R China 2.Beijing Wenge Technol Co Ltd, Beijing 100000, Peoples R China 3.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Fang, Xing,Hao, YanNi,Luo, Yin,et al. Forged facial video detection framework based on multi-region temporal relationship feature[J]. AIP ADVANCES,2023,13(8):8.
APA	Fang, Xing,Hao, YanNi,Luo, Yin,Xu, Nan,&Cao, Jia.(2023).Forged facial video detection framework based on multi-region temporal relationship feature.AIP ADVANCES,13(8),8.
MLA	Fang, Xing,et al."Forged facial video detection framework based on multi-region temporal relationship feature".AIP ADVANCES 13.8(2023):8.