Knowledge Commons of Institute of Automation,CAS
StyTr2: Image Style Transfer with Transformers | |
Deng, Yingying1,2; Tang, Fan3; Dong, Weiming1,2; Ma, Chongyang4; Pan, Xingjia2; Wang, Lei5; Xu, Changsheng1,2 | |
2022 | |
会议名称 | IEEE Conference on Computer Vision and Pattern Recognition |
会议日期 | 2022-6 |
会议地点 | New Orleans, Louisiana |
摘要 | The goal of image style transfer is to render an image with artistic features guided by a style reference while maintaining the original content. Owing to the locality in convolutional neural networks (CNNs), extracting and maintaining the global information of input images is difficult. Therefore, traditional neural style transfer methods face biased content representation. To address this critical issue, we take long-range dependencies of input images into account for image style transfer by proposing a transformer-based approach called StyTr$^2$. In contrast with visual transformers for other vision tasks, StyTr$^2$ contains two different transformer encoders to generate domain-specific sequences for content and style, respectively. Following the encoders, a multi-layer transformer decoder is adopted to stylize the content sequence according to the style sequence. We also analyze the deficiency of existing positional encoding methods and propose the content-aware positional encoding (CAPE), which is scale-invariant and more suitable for image style transfer tasks. Qualitative and quantitative experiments demonstrate the effectiveness of the proposed StyTr$^2$ compared with state-of-the-art CNN-based and flow-based approaches. Code and models are available at https://github.com/diyiiyiii/StyTR-2. |
文献类型 | 会议论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/48627 |
专题 | 多模态人工智能系统全国重点实验室_多媒体计算 |
通讯作者 | Tang, Fan; Dong, Weiming |
作者单位 | 1.School of Artificial Intelligence, UCAS 2.NLPR, Institute of Automation, CAS 3.School of Artificial Intelligence, Jilin University 4.Kuaishou Technology 5.CIPUC |
第一作者单位 | 模式识别国家重点实验室 |
通讯作者单位 | 模式识别国家重点实验室 |
推荐引用方式 GB/T 7714 | Deng, Yingying,Tang, Fan,Dong, Weiming,et al. StyTr2: Image Style Transfer with Transformers[C],2022. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
Deng_StyTr2_Image_St(10025KB) | 会议论文 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论