Knowledge Commons of Institute of Automation,CAS
Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network With Graph Representation Learning | |
Qi, Xingqun1,2,3; Sun, Muyi4,5; Wang, Zijian6; Liu, Jiaming7; Li, Qi4; Zhao, Fang8; Zhang, Shanghang7; Shan, Caifeng8,9 | |
发表期刊 | IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS |
ISSN | 2162-237X |
2023-12-19 | |
页码 | 14 |
摘要 | Biphasic face photo-sketch synthesis has significant practical value in wide-ranging fields such as digital entertainment and law enforcement. Previous approaches directly generate the photo-sketch in a global view, they always suffer from the low quality of sketches and complex photograph variations, leading to unnatural and low-fidelity results. In this article, we propose a novel semantic-driven generative adversarial network to address the above issues, cooperating with graph representation learning. Considering that human faces have distinct spatial structures, we first inject class-wise semantic layouts into the generator to provide style-based spatial information for synthesized face photographs and sketches. In addition, to enhance the authenticity of details in generated faces, we construct two types of representational graphs via semantic parsing maps upon input faces, dubbed the intraclass semantic graph (IASG) and the interclass structure graph (IRSG). Specifically, the IASG effectively models the intraclass semantic correlations of each facial semantic component, thus producing realistic facial details. To preserve the generated faces being more structure-coordinated, the IRSG models interclass structural relations among every facial component by graph representation learning. To further enhance the perceptual quality of synthesized images, we present a biphasic interactive cycle training strategy by fully taking advantage of the multilevel feature consistency between the photograph and sketch. Extensive experiments demonstrate that our method outperforms the state-of-the-art competitors on the CUHK Face Sketch (CUFS) and CUHK Face Sketch FERET (CUFSF) datasets. |
关键词 | Face photo-sketch synthesis generative adversarial network graph representation learning intraclass and interclass iterative cycle training (ICT) |
DOI | 10.1109/TNNLS.2023.3341246 |
收录类别 | SCI |
语种 | 英语 |
资助项目 | Talent Introduction Program for Youth Innovation Teams of Shandong Province |
项目资助者 | Talent Introduction Program for Youth Innovation Teams of Shandong Province |
WOS研究方向 | Computer Science ; Engineering |
WOS类目 | Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic |
WOS记录号 | WOS:001130341400001 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
是否为代表性论文 | 否 |
七大方向——子方向分类 | 生物特征识别 |
国重实验室规划方向分类 | 视觉信息处理 |
是否有论文关联数据集需要存交 | 否 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/54926 |
专题 | 多模态人工智能系统全国重点实验室 |
通讯作者 | Shan, Caifeng |
作者单位 | 1.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 2.Peking Univ, Sch Comp Sci, Beijing 100871, Peoples R China 3.Hong Kong Univ Sci & Technol, Acad Interdisciplinary Studies, Hong Kong, Peoples R China 4.Chinese Acad Sci, Inst Automat, NLPR, CRIPAC, Beijing 100190, Peoples R China 5.Beijing Univ Posts & Telecommun, Sch Artificial Intelligence, Beijing 100876, Peoples R China 6.Univ Sydney, Sch Comp Sci, Sydney, NSW 2008, Australia 7.Peking Univ, Sch Comp Sci, Natl Key Lab Multimedia Informat Proc, Beijing 100871, Peoples R China 8.Nanjing Univ, Sch Intelligence Sci & Technol, Nanjing 210023, Peoples R China 9.Shandong Univ Sci & Technol, Coll Elect Engn & Automat, Qingdao 266590, Peoples R China |
第一作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Qi, Xingqun,Sun, Muyi,Wang, Zijian,et al. Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network With Graph Representation Learning[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2023:14. |
APA | Qi, Xingqun.,Sun, Muyi.,Wang, Zijian.,Liu, Jiaming.,Li, Qi.,...&Shan, Caifeng.(2023).Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network With Graph Representation Learning.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,14. |
MLA | Qi, Xingqun,et al."Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network With Graph Representation Learning".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2023):14. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
Biphasic_Face_Photo-(6718KB) | 期刊论文 | 出版稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论