Semantic 3D-Aware Portrait Synthesis and Manipulation Based on Compositional Neural Radiance Field | |
Tianxiang Ma1,2; Bingchuan Li3; Qian He3; Jing Dong2; Tieniu Tan2,4 | |
2023 | |
会议名称 | AAAI Conference on Artificial Intelligence |
会议日期 | 2.7-2.14 |
会议地点 | 美国华盛顿 |
摘要 | Recently 3D-aware GAN methods with neural radiance field have developed rapidly. However, current methods model the whole image as an overall neural radiance field, which limits the partial semantic editability of synthetic results. Since NeRF renders an image pixel by pixel, it is possible to split NeRF in the spatial dimension. We propose a Compositional Neural Radiance Field (CNeRF) for semantic 3D-aware portrait synthesis and manipulation. CNeRF divides the image by semantic regions and learns an independent neural radiance field for each region, and finally fuses them and renders the complete image. Thus we can manipulate the synthesized semantic regions independently, while fixing the other parts unchanged. Furthermore, CNeRF is also designed to decouple shape and texture within each semantic region. Compared to state-of-the-art 3D-aware GAN methods, our approach enables fine-grained semantic region manipulation, while maintaining high-quality 3D-consistent synthesis. The ablation studies show the effectiveness of the structure and loss function used by our method. In addition real image inversion and cartoon portrait 3D editing experiments demonstrate the application potential of our method. |
七大方向——子方向分类 | 图像视频处理与分析 |
国重实验室规划方向分类 | 视觉信息处理 |
是否有论文关联数据集需要存交 | 否 |
文献类型 | 会议论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/56661 |
专题 | 智能感知与计算研究中心 |
通讯作者 | Jing Dong |
作者单位 | 1.School of Artificial Intelligence, University of Chinese Academy of Sciences 2.CRIPAC & NLPR, Institute of Automation, Chinese Academy of Sciences 3.ByteDance Ltd, Beijing, China 4.Nanjing University |
第一作者单位 | 模式识别国家重点实验室 |
通讯作者单位 | 模式识别国家重点实验室 |
推荐引用方式 GB/T 7714 | Tianxiang Ma,Bingchuan Li,Qian He,et al. Semantic 3D-Aware Portrait Synthesis and Manipulation Based on Compositional Neural Radiance Field[C],2023. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
Semantic 3D-Aware Po(4156KB) | 会议论文 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论