CASIA OpenIR  > 智能感知与计算
A Unified Framework for High Fidelity Face Swap and Expression Reenactment
Peng, Bo1,2; Fan, Hongxing1; Wang, Wei1; Dong, Jing1; Lyu, Siwei3
Source PublicationIEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
ISSN1051-8215
2022-06-01
Volume32Issue:6Pages:3673-3684
Corresponding AuthorDong, Jing(jdong@nlpr.ia.ac.cn)
AbstractFace manipulation techniques improve fast with the development of powerful image generation models. Two particular face manipulation methods, namely face swap and expression reenactment attract much attention for their flexibility and ease to generate high quality synthesis results. Recently, these two subjects are actively studied. However, most existing methods treat the two tasks separately, ignoring their underlying similarity. In this paper, we propose to tackle the two problems within a unified framework that achieves high quality synthesis results. The enabling component for our unified framework is the clean disentanglement of 3D pose, shape, and expression factors and then recombining them for different tasks accordingly. We then use the same set of 2D representations for face swap and expression reenactment tasks that are input to a common image translation model to directly generate the final synthetic images. Once trained, the proposed model can accomplish both face swap and expression reenactment tasks for previously unseen subjects. Comprehensive experiments and comparisons show that the proposed method achieves high fidelity results in multiple aspects, and it is especially good at faithfully preserving source facial shape in the face swap task, and accurately transferring facial movements in the expression reenactment task.
KeywordFaces Task analysis Videos Shape Three-dimensional displays Face recognition Information integrity Face swap expression reenactment 3DMM video manipulation
DOI10.1109/TCSVT.2021.3106047
Indexed BySCI
Language英语
Funding ProjectNational Natural Science Foundation of China (NSFC)[61902400] ; National Natural Science Foundation of China (NSFC)[U19B2038] ; National Natural Science Foundation of China (NSFC)[61972395] ; National Natural Science Foundation of China (NSFC)[61772529] ; National Natural Science Foundation of China (NSFC)[62076240] ; Beijing Natural Science Foundation[4192058]
Funding OrganizationNational Natural Science Foundation of China (NSFC) ; Beijing Natural Science Foundation
WOS Research AreaEngineering
WOS SubjectEngineering, Electrical & Electronic
WOS IDWOS:000805833400031
PublisherIEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Citation statistics
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/49585
Collection智能感知与计算
Corresponding AuthorDong, Jing
Affiliation1.Chinese Acad Sci, Inst Automat, Ctr Res Intelligent Percept & Comp CRIPAC, Natl Lab Pattern Recognit NLPR, Beijing 100190, Peoples R China
2.Chinese Acad Sci, Inst Informat Engn, State Key Lab Informat Secur, Beijing 100093, Peoples R China
3.Univ Buffalo State Univ New York Buffalo, Sch Engn & Appl Sci, Buffalo, NY 14260 USA
First Author AffilicationChinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China
Corresponding Author AffilicationChinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China
Recommended Citation
GB/T 7714
Peng, Bo,Fan, Hongxing,Wang, Wei,et al. A Unified Framework for High Fidelity Face Swap and Expression Reenactment[J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,2022,32(6):3673-3684.
APA Peng, Bo,Fan, Hongxing,Wang, Wei,Dong, Jing,&Lyu, Siwei.(2022).A Unified Framework for High Fidelity Face Swap and Expression Reenactment.IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,32(6),3673-3684.
MLA Peng, Bo,et al."A Unified Framework for High Fidelity Face Swap and Expression Reenactment".IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 32.6(2022):3673-3684.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Peng, Bo]'s Articles
[Fan, Hongxing]'s Articles
[Wang, Wei]'s Articles
Baidu academic
Similar articles in Baidu academic
[Peng, Bo]'s Articles
[Fan, Hongxing]'s Articles
[Wang, Wei]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Peng, Bo]'s Articles
[Fan, Hongxing]'s Articles
[Wang, Wei]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.