CASIA OpenIR  > 多模态人工智能系统全国重点实验室  > 三维可视计算
基于单张RGB 图像的人脸三维精细建模研究
杨明鑫
2021-05
Pages62
Subtype硕士
Abstract

请输入中文摘要三维人脸重建是计算机视觉和计算机图形学领域中的一个重要研究课题。根
据输入数据的不同,三维人脸重建的方法可以被分为多个种类,而本文主要研
究基于单张二维图像的三维人脸重建。由于基于单张图像的三维人脸重建是一
个高度信息缺失问题,先前的工作主要提出一种人脸参数模型来简化重建工作。
后续的很多研究也直接利用了人脸参数模型作为中间媒介来进行三维人脸重建。
但是,三维人脸参数模型本身具有无法捕捉人脸细节的缺陷,所以该类方法的重
建结果无法直接用于很多需要高保真重建的实际应用。为解决该问题,我们基于
不同的观察,分别从两个方面来补充人脸细节,并提出了相对应的两个子研究任
务,即分别从表情驱动的几何细节生成以及基于先验纹理的纹理细节生成的角
度来完成三维人脸精细重建工作。论文主要的研究内容和贡献总结如下:
1. 基于表情驱动的人脸几何细节生成
本工作提出了一种基于表情驱动的三维人脸细节恢复方法,主要采用了从
粗糙到精细的范式:首先采用人脸参数模型回归方法重建一个基础的粗粒度模
型,而后根据表情特征以及人脸属性特征生成在粗粒度人脸模型之上的表情驱
动的几何细节。其中,表情表征是利用了人脸参数模型中表情参数解码后所得到
的位移图,而人脸个性化属性表征则是根据图像通过一个编码解码网络获取的
表征。而后,我们利用了分块动态卷积作为结合这两个表征的模块以生成细节
补充图。该工作针对于传统线性三维人脸参数模型中的细节缺失问题做了补充,
并利用了表情表征作为这些人脸细节的生成线索,以达成了高精度的三维人脸
几何重建。
2. 可直接渲染的自监督三维人脸精细纹理重建
本工作提出了一个基于单张人脸图像的精细三维人脸纹理重建工作。其提
出主要着眼于解决以下两个问题:第一,对于自然图像中的遮挡人脸和非正面人
脸的纹理重建问题;第二,对于复杂环境光照下可直接渲染的人脸细节纹理重建
问题。为了解决这两个问题,本工作首先结合了生成式人脸纹理恢复方法以及重
建式纹理恢复方法,利用生成的人脸纹理的光照解耦特性以及完整性,指导重
建式纹理恢复模块以获取可直接渲染的、高精度的人脸纹理。在此之上,本工作
设计了专门的光照解耦损失函数以及正则项损失来保证重建人脸纹理的高质量。
此外,为了应对复杂的环境光照,本工作还提出了全新的细节光照表征方法以增
强对于环境光照的捕捉能力。

Other Abstract

3D face reconstruction is an important topic in computer vision and computer
graphics. It can be divided into many categories according to the input mode. This paper
mainly focuses on the most difficult one which conducts 3D face reconstruction based
on a single face image. Since single RGB inputs lacks of information highly, previous
work has proposed face parametric model to simplify the reconstruction. Many subsequent
works also directly utilize the face parametric model as an intermediate proxy
model to reconstruct the 3D face. However, the 3D face parametric model itself has
the defect that it cannot capture the details of the face, and the reconstruction results
based on it cannot be directly applied to many downstream applications requiring highfidelity
reconstruction. This is also the main problem which we expect to solve in this
paper. We try to complement the details of the face from two aspects and different observations
respectively. According to this, we propose two sub works, which focus on
reconstructing the geometric facial details from expression and reconstructing detailed
facial texture from a prior texture respectively.
1. Expression driven human facial geometric detail generation
In this work, we propose a expression driven 3d facial detail recovery method. It
mainly takes the coarse-to-fine paradigm that we first utilize a facial parametric model
regression approach to acquire a basic 3d coarse face model and then leverage expression
feature representation and the human facial attribute feature to generate geometric facial
details upon the coarse face model. The expression feature representation adopted
in our pipeline is the expression displacement map decoded from 3DMM expression
parameters. And we leverage an encoder-decoder network to obtain the human face attribute
representation. Then, a region-based dynamic convolutional layer is utilized as
the merging layer of these two features and produce the detail displacement map. Our
wok complement the lacking detail problem of traditional 3DMM and take advantage
of the facial expression as the guided clue to achieve the high-quality detailed 3d facial
geometry reconstruction.
2. Self-supervised Rerenderable and detailed 3d facial texture reconstruction
In this work, we propose an approach to reconstruct detailed human facial texture
from a single image. It mainly focuses on resolving two problems existed in the mainstream
study; the first problem is the non-frontal and occluded faces that are commonly
encountered in the in-the-wild facial image data. This is not fully worked our in the
state-of-the-art researches. And the second issue is considering the spatially-complex
environmental illumination which could not be resolved by the commonly-used Spherical
Harmonics representation. Accordingly, we propose to merge the generative methods
and reconstruction methods utilizing the completeness and disentanglement with
illumination of generated texture to guide the reconstruction method to achieve detailed
and directly-rerenderable human texture. In addition, we devise specialized illumination
disentanglement loss and regularization loss to guarantee the high-quality reconstruction.
Besides, we also propose a brand new detailed illumination representation to
enhance the capacity to capture the complex environmental illumination.

Keyword三维重建 人脸重建 几何细节 纹理细节 三维人脸精细重建
Language中文
Sub direction classification计算机图形学与虚拟现实
Document Type学位论文
Identifierhttp://ir.ia.ac.cn/handle/173211/44798
Collection多模态人工智能系统全国重点实验室_三维可视计算
Corresponding Author杨明鑫
Recommended Citation
GB/T 7714
杨明鑫. 基于单张RGB 图像的人脸三维精细建模研究[D]. 中国科学院自动化研究所. 中国科学院自动化研究所,2021.
Files in This Item:
File Name/Size DocType Version Access License
final_thesis.pdf(9203KB)学位论文 开放获取CC BY-NC-SA
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[杨明鑫]'s Articles
Baidu academic
Similar articles in Baidu academic
[杨明鑫]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[杨明鑫]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.