毕业生知识库

随着信息技术和智能技术的快速发展，图像、视频编辑工具越来越智能化、简易化，给人们生活带来方便的同时，对视觉内容的真实可信性带来巨大挑战。同时，社交网络的广泛普及更是起到推波助澜的作用，大大加剧其危害性。因此，本论文中，将针对该问题开展取证研究工作。网络上传播的图像和视频，经过多次转载和相应操作，使得传统的取证方法基本失效。本论文从对成像场景的研究入手，充分利用辅助信息，研究成像过程中的几何约束、物理约束、生命体指征评估等理论与方法，针对危害性较大的人像篡改伪造问题，提出多种取证方法，以期解决视觉内容真实性取证研究中的鲁棒性和准确性问题，推动取证技术走向实际应用。本论文的研究内容包括以下方面：

1. 基于几何透视失真的人脸图像拼接检测

从三维物体在不同相机拍摄参数下呈现不同透视失真特性这一固有现象出发，首次提出将人脸图像的失真特性与相机内参数之间的不一致性作为新的取证线索。研究了基于三维人脸模型的相机拍摄参数估计问题。不同于经典相机标定问题，人脸关键点定位具有较高误差和不确定性。因此，本文首先利用三维和二维人脸关键点对应关系粗略估计相机参数，进而利用人脸轮廓进行微调优化，以获得更高的估计准确度。最终通过度量相机内参数之间的不一致性检测人像篡改。

2. 基于平面接触几何约束的人像拼接检测

首次探索将三维物体底面与其支撑平面之间的共面接触这一几何约束作为新的取证线索。二维图像内简单的物体拼接很难保证其在三维场景中与支撑平面准确接触。本文在研究不规则形状物体的三维几何信息恢复的基础上，提出利用三维形状统计模型从单张图像中估计物体拍摄姿态的算法，并利用场景中的消隐线估计支撑平面的法向量，最终在姿态空间中度量多个物体与支撑平面之间的姿态平行度，以揭示篡改行为。本文将该框架应用于人像拼接取证，使其能够有效检测“悬浮脚”等篡改事件。

3. 基于光照一致性的人脸图像拼接检测

在重建部分场景几何信息的基础上，研究基于光照反射模型的场景三维光照环境估计算法，利用光照方向一致性物理约束检测人脸图像拼接。原有的基于三维光照环境的取证工作中，对人脸模型做出过强假设，限制了算法的准确性和适用性。本文通过充分挖掘利用三维人脸模型的几何遮挡信息和纹理信息对之前的强假设进行放松，提出一般化光照反射模型，从而得到更准确的光照估计结果和更强的取证判别力。进一步通过单张图像拟合三维人脸模型的方法实现了整个取证过程的自动化，增强了取证算法的实用性。

4. 基于脉搏信号估计的虚假人脸视频检测

人脸表面的光照反射强度是随着时间周期性变化的，这是因为人脸的肤色随着心跳有着周期性的变化，本文正是利用该特性来进行虚假人脸视频的检测。研究人脸视频中的脉搏信号提取算法，以脉搏信号的有无为依据对人脸视频的真实性进行判别。通过对已有工作的缺陷进行分析，发现信号频域分辨率不足和算法参数的敏感性制约了方法的有效性。提出延长视频片段时间以获得足够的信号频域分辨率，并采用基于色度成分的脉搏信号提取算法，以克服由于视频片段增长带来的运动鲁棒性问题。最终通过分析信号频域特征对视频中人脸的真实性做出判断。在计算机合成人脸以及假体面具人脸的视频上验证了所提方法的有效性。

Together with the fast development of information science and intelligent technology, the editing of images and videos becomes more convenient. It also brings serious challenges to the authenticity and credibility of visual contents apart from the goodness to everyday life. Meanwhile, the widely popularized social networks help the fast spread of false information, which further increases the impacts. This thesis investigates the forensic authentication of visual contents to counter these serious situations. Traditional forensic methods almost fail to detect forgeries after images and videos are circulated around on the web. This thesis focuses on the investigation of imaging scene clues. We make full use of available prior knowledge and explore the theory of forensics on geometric constraints, physical constraints and physiological signs. Targeted at the harmful forgeries of people’s portraits, this thesis proposes multiple forensic methods to solve the problem of low robustness and accuracy and hope to advance practical application of forensics. This thesis includes the following research aspects:

1. Detection of Face Image Splicing Based on Geometric Perspective Distortion There is an intrinsic property that imaged 3D objects have different characteristics of perspective distortion when using different camera parameters. We propose a new forensic clue which is the inconsistency between observed perspective distortion and the camera parameter. The designed forensic method estimates camera parameters from a single questioned face image with the help of accurate 3D face model. Our problem is more challenging than classical camera calibration, because the localization of facial landmarks has more error and uncertainty. We propose to add a refinement step based on contour observations after the coarse estimation of camera parameters using 3D and 2D facial landmark correspondences. This strategy is proved capable of more accurate estimation. The final forensic decision is made by measuring the distance between estimated and claimed camera intrinsic parameters.

2. Portrait Splicing Detection Based on Geometric Constraints of Planar Contact

We propose to use the geometric co-planar constraints in 3D objects contacting their supporting plane as a new forensic clue. The simple splicing manipulation in 2D image cannot ensure that spliced parts precisely contact the supporting plane in the underlying 3D scene. We investigate the recovery of geometric information for imaged 3D objects of unknown shape. We propose to jointly estimate the pose and shape of 3D objects from a single image based on a statistical 3D shape model, and to estimate the normal vector of supporting plane based on its vanishing line. The parallelism between multiple objects’ base planes and the supporting plane is measured to determine the authenticity. We apply this forensic framework to the splicing detection of portraits of people and find it effective in detecting the “floating feet” artifact.

3. Detection of Face Image Splicing Based on Lighting Consistency

Based on our last two research work, we first reconstruct partial scene geometry and then estimate the lighting environment in the scene using a light reflection model. The physical constraint of lighting environment consistency can be used as a forensic clue. Previous work on lighting environment based forensics makes too strong assumptions for faces, which restricts the accuracy and applicability. We make full usage of the geometric occlusion and facial texture information of the 3D face model to relax two previous assumptions. The proposed generalized light reflection model can obtain more accurate lighting estimation results and stronger forensic performance. We also automate the whole process and improve the practicality by developing a fitting method to reconstruct face from a single unconstrained image.

4. Detection of Fake Faces in Videos Based on Pulse Signal Estimation

The light intensity reflected from facial skin fluctuates with the pulsation of heart beats. We extract this pulse signal from videos of faces, and judge the authenticity of faces based on the existence of this signal. The flaws in existing work are the short video clip and its sensitivity to parameters, which restrict its effectiveness. We propose to extend the clip duration to obtain adequate frequency resolution, and to adopt a chrominance based pulse signal extraction method that is more robust to motion artifact. The frequency characteristic of extracted signal is analyzed to decide the authenticity of examined face. We prove the effectiveness of the proposed method by conducting experiments on computer generated faces and fake 3D face masks.