Depth assisted novel view synthesis using few images

doi:10.1016/j.imavis.2024.105079

	Depth assisted novel view synthesis using few images
	Li, Qian 1; Fu, Rao 1; Tang, Fulin2
发表期刊	IMAGE AND VISION COMPUTING
ISSN	0262-8856
	2024-07-01
卷号	147 页码:9
通讯作者	Fu, Rao(rao.fu@inria.fr)
摘要	In this paper, we introduce a novel approach to improve the performance of Neural Radiance Fields (NeRF) from limited input views. NeRF has exhibited impressive capabilities in producing photo-realistic renderings when trained on dense input views, but its performance degrades as the number of training views decreases. Our key insight is that the original NeRF lacks geometric regularization and appearance information due to limited inputs, resulting in an over-fitting issue. To address this challenge, we present a novel method: first, a global sampling method with geometric regularization is employed by utilizing warped images as additional pseudoviews, which optimizes the multi-view consistency during the training. Second, we introduce a local patch sampling technique with perceptual regularization to ensure pixel correspondence in appearance. Furthermore, we incorporate depth information for explicit geometry regularization. We evaluate our method on the DTU dataset and LLFF dataset from a different number of inputs. Extensive evaluations demonstrate that our approach outperforms existing benchmarks across various metrics, achieving state-of-the-art results.
关键词	Neural radiance fields View synthesis Image warping
DOI	10.1016/j.imavis.2024.105079
收录类别	SCI
语种	英语
WOS研究方向	Computer Science ; Engineering ; Optics
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Software Engineering ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic ; Optics
WOS记录号	WOS:001244560700001
出版者	ELSEVIER
引用统计
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/58702
专题	多模态人工智能系统全国重点实验室_机器人视觉
通讯作者	Fu, Rao
作者单位	1.Inria, Le Chesnay Rocquencourt, France 2.Chinese Acad Sci, Inst Automat, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Li, Qian,Fu, Rao,Tang, Fulin. Depth assisted novel view synthesis using few images[J]. IMAGE AND VISION COMPUTING,2024,147:9.
APA	Li, Qian,Fu, Rao,&Tang, Fulin.(2024).Depth assisted novel view synthesis using few images.IMAGE AND VISION COMPUTING,147,9.
MLA	Li, Qian,et al."Depth assisted novel view synthesis using few images".IMAGE AND VISION COMPUTING 147(2024):9.