Knowledge Commons of Institute of Automation,CAS
W-Net: Structure and Texture Interaction for Image Inpainting | |
Zhang, Ruisong1,2; Quan, Weize1,2; Zhang, Yong3; Wang, Jue3; Yan, Dong-Ming1,2 | |
发表期刊 | IEEE Transactions on Multimedia |
ISSN | 1520-9210 |
2022-11 | |
卷号 | 25页码:1-12 |
通讯作者 | Yan, Dong-Ming(yandongming@gmail.com) |
摘要 | Recent literature has developed two advanced tools for image inpainting: appearance propagation and attention matching. However, given the ineffective feature reorganization and vulnerable attention maps, existing works yield suboptimal results with distorted structures and inconsistent contents. Furthermore, we observe that deep sampling layers (DSL) and shallow skip connections (SSC) in U-Net separately promote image structure inference and texture synthesis. To address the above two issues, we devise a W-shaped network (W-Net), which consists of two key components: a texture spatial attention (TSA) module in SSC and a structure channel excitation (SCE) module in DSL. W-Net is a two-stage network, with coarse and refined structures derived at each stage. Meanwhile, the TSA module fills incomplete textures with reliable attention scores under the guidance of coarse structures, which effectively diminishes inconsistency from appearance to semantics. The SCE module rectifies structures according to the difference between coarse structures and refined structures enhanced by texture features. Then the module motivates them to produce more reasonable shapes. Complete textures and refined structures constitute desired inpainted images, as the output of W-Net. Experiments on multiple datasets demonstrate the superior performance of W-Net. The source code is available at https://github.com/Evergrow/W-Net |
关键词 | Index Terms-Image inpainting structure and texture convolutional neural network attention |
DOI | 10.1109/TMM.2022.3219728 |
收录类别 | SCI |
语种 | 英语 |
资助项目 | National Natural Science Foundation of China[62102418] ; National Natural Science Foundation of China[62172415] ; Tencent AI Laboratory Rhino-Bird Focused Research Program[JR202127] ; Open Project Program of National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University[2021SCUVS002] ; Open Research Fund Program of State key Laboratory of Hydroscience and Engineering, Tsinghua University[sklhse-2022-D-04] |
项目资助者 | National Natural Science Foundation of China ; Tencent AI Laboratory Rhino-Bird Focused Research Program ; Open Project Program of National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University ; Open Research Fund Program of State key Laboratory of Hydroscience and Engineering, Tsinghua University |
WOS研究方向 | Computer Science ; Telecommunications |
WOS类目 | Computer Science, Information Systems ; Computer Science, Software Engineering ; Telecommunications |
WOS记录号 | WOS:001102654000043 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
七大方向——子方向分类 | 计算机图形学与虚拟现实 |
国重实验室规划方向分类 | 多模态协同认知 |
是否有论文关联数据集需要存交 | 否 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/51500 |
专题 | 多模态人工智能系统全国重点实验室 |
通讯作者 | Yan, Dong-Ming |
作者单位 | 1.NLPR, Institute of Automation, Chinese Academy of Science 2.School of Artificial Intelligence, the University of Chinese Academy of Sciences 3.Tencent AI Lab, ShenZhen |
第一作者单位 | 模式识别国家重点实验室 |
通讯作者单位 | 模式识别国家重点实验室 |
推荐引用方式 GB/T 7714 | Zhang, Ruisong,Quan, Weize,Zhang, Yong,et al. W-Net: Structure and Texture Interaction for Image Inpainting[J]. IEEE Transactions on Multimedia,2022,25:1-12. |
APA | Zhang, Ruisong,Quan, Weize,Zhang, Yong,Wang, Jue,&Yan, Dong-Ming.(2022).W-Net: Structure and Texture Interaction for Image Inpainting.IEEE Transactions on Multimedia,25,1-12. |
MLA | Zhang, Ruisong,et al."W-Net: Structure and Texture Interaction for Image Inpainting".IEEE Transactions on Multimedia 25(2022):1-12. |
条目包含的文件 | 条目无相关文件。 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论