Robust Local Light Field Synthesis via Occlusion-aware Sampling and Deep Visual Feature Fusion

doi:10.1007/s11633-022-1381-9

CASIA OpenIR > 学术期刊 > Machine Intelligence Research

	Robust Local Light Field Synthesis via Occlusion-aware Sampling and Deep Visual Feature Fusion
	Wenpeng Xing 1; Jie Chen 1; Yike Guo 1,2
发表期刊	Machine Intelligence Research
ISSN	2731-538X
	2023
卷号	20 期号:3 页码:408-420
摘要	Novel view synthesis has attracted tremendous research attention recently for its applications in virtual reality and immersive telepresence. Rendering a locally immersive light field (LF) based on arbitrary large baseline RGB references is a challenging problem that lacks efficient solutions with existing novel view synthesis techniques. In this work, we aim at truthfully rendering local immersive novel views/LF images based on large baseline LF captures and a single RGB image in the target view. To fully explore the precious information from source LF captures, we propose a novel occlusion-aware source sampler (OSS) module which efficiently transfers the pixels of source views to the target view's frustum in an occlusion-aware manner. An attention-based deep visual fusion module is proposed to fuse the revealed occluded background content with a preliminary LF into a final refined LF. The proposed source sampling and fusion mechanism not only helps to provide information for occluded regions from varying observation angles, but also proves to be able to effectively enhance the visual rendering quality. Experimental results show that our proposed method is able to render high-quality LF images/novel views with sparse RGB references and outperforms state-of-the-art LF rendering and novel view synthesis methods.
关键词	Novel view synthesis, light field (LF) imaging, multi-view stereo, occlusion sampling, deep visual feature (DVF) fusion
DOI	10.1007/s11633-022-1381-9
引用统计	被引频次：2[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/55987
专题	学术期刊_Machine Intelligence Research
作者单位	1.Department of Computer Science, Hong Kong Baptist University, Hong Kong 999077, China 2.Data Science Institute, Imperial College London, London SW7 2AZ, UK
推荐引用方式 GB/T 7714	Wenpeng Xing,Jie Chen,Yike Guo. Robust Local Light Field Synthesis via Occlusion-aware Sampling and Deep Visual Feature Fusion[J]. Machine Intelligence Research,2023,20(3):408-420.
APA	Wenpeng Xing,Jie Chen,&Yike Guo.(2023).Robust Local Light Field Synthesis via Occlusion-aware Sampling and Deep Visual Feature Fusion.Machine Intelligence Research,20(3),408-420.
MLA	Wenpeng Xing,et al."Robust Local Light Field Synthesis via Occlusion-aware Sampling and Deep Visual Feature Fusion".Machine Intelligence Research 20.3(2023):408-420.

条目包含的文件		下载所有文件
文件名称/大小	文献类型	版本类型	开放类型	使用许可
MIR-2022-06-204.pdf（2263KB）	期刊论文	出版稿	开放获取	CC BY-NC-SA	浏览下载