Self-Prior Guided Pixel Adversarial Networks for Blind Image Inpainting

doi:10.1109/TPAMI.2023.3284431

	Self-Prior Guided Pixel Adversarial Networks for Blind Image Inpainting
	Wang, Juan 1; Yuan, Chunfeng1 ; Li, Bing1,2 ; Deng, Ying 3; Hu, Weiming1,4,5 ; Maybank, Stephen 6
发表期刊	IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
ISSN	0162-8828
	2023-10-01
卷号	45 期号:10 页码:12377-12393
通讯作者	Li, Bing(bli@nlpr.ia.ac.cn)
摘要	Blind image inpainting involves two critical aspects, i.e., "where to inpaint" and "how to inpaint". Knowing "where to inpaint" can eliminate the interference arising from corrupted pixel values; a good "how to inpaint" strategy yields high-quality inpainted results robust to various corruptions. In existing methods, these two aspects usually lack explicit and separate consideration. This paper fully explores these two aspects and proposes a self-prior guided inpainting network (SIN). The self-priors are obtained by detecting semantic-discontinuous regions and by predicting global semantic structures of the input image. On the one hand, the self-priors are incorporated into the SIN, which enables the SIN to perceive valid context information from uncorrupted regions and to synthesize semantic-aware textures for corrupted regions. On the other hand, the self-priors are reformulated to provide a pixel-wise adversarial feedback and a high-level semantic structure feedback, which can promote the semantic continuity of inpainted images. Experimental results demonstrate that our method achieves state-of-the-art performance in metric scores and in visual quality. It has an advantage over many existing methods that assume "where to inpaint" is known in advance. Extensive experiments on a series of related image restoration tasks validate the effectiveness of our method in obtaining high-quality inpainting.
关键词	Blind image inpainting semantic-discontinuity detection layout map prediction pixel generative adversarial network
DOI	10.1109/TPAMI.2023.3284431
关键词[WOS]	COMPRESSION
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China ; Natural Science Foundation of China[2020AAA0106800] ; Natural Science Foundation of China[62202470] ; Natural Science Foundation of China[61972397] ; Natural Science Foundation of China[62122086] ; Natural Science Foundation of China[U1936204] ; Natural Science Foundation of China[62036011] ; Natural Science Foundation of China[62192782] ; Natural Science Foundation of China[61721004] ; Beijing Natural Science Foundation[U2033210] ; Beijing Natural Science Foundation[4224093] ; Beijing Natural Science Foundation[JQ21017] ; Major Projects of Guangdong Education Department for Foundation Research and Applied Research[L223003] ; Major Projects of Guangdong Education Department for Foundation Research and Applied Research[2017KZDXM081] ; Guangdong Provincial University Innovation Team Project[2018KZDXM066] ; Youth Innovation Promotion Association, CAS ; [2020KCXTD045]
项目资助者	National Key Research and Development Program of China ; Natural Science Foundation of China ; Beijing Natural Science Foundation ; Major Projects of Guangdong Education Department for Foundation Research and Applied Research ; Guangdong Provincial University Innovation Team Project ; Youth Innovation Promotion Association, CAS
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:001068816800054
出版者	IEEE COMPUTER SOC
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/53069
专题	多模态人工智能系统全国重点实验室
通讯作者	Li, Bing
作者单位	1.Chinese Acad Sci CASIA, Inst Automat, State Key Lab Multimodal Artificial Intelligence S, Beijing 100190, Peoples R China 2.People AI Inc, Beijing 100080, Peoples R China 3.Nanchang Hangkong Univ, Sch Aeronaut Mfg Engn, Nanchang 330063, Peoples R China 4.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100190, Peoples R China 5.Shanghai Tech Univ, Sch Informat Sci & Technol, Shanghai 201210, Peoples R China 6.Birkbeck Coll, Dept Comp Sci & Informat Syst, London WC1E 7HX, England
第一作者单位	中国科学院自动化研究所
通讯作者单位	中国科学院自动化研究所
推荐引用方式 GB/T 7714	Wang, Juan,Yuan, Chunfeng,Li, Bing,et al. Self-Prior Guided Pixel Adversarial Networks for Blind Image Inpainting[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2023,45(10):12377-12393.
APA	Wang, Juan,Yuan, Chunfeng,Li, Bing,Deng, Ying,Hu, Weiming,&Maybank, Stephen.(2023).Self-Prior Guided Pixel Adversarial Networks for Blind Image Inpainting.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,45(10),12377-12393.
MLA	Wang, Juan,et al."Self-Prior Guided Pixel Adversarial Networks for Blind Image Inpainting".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 45.10(2023):12377-12393.