Optical flow-assisted multi-level fusion network for Light Field image angular reconstruction

doi:10.1016/j.image.2023.117031

	Optical flow-assisted multi-level fusion network for Light Field image angular reconstruction
	Liu, Deyang 1,2,4; Mao, Yifan 1; Huang, Yan 3; Cao, Liqun 1; Wang, Yuanzhi 1; Fang, Yuming 2
发表期刊	SIGNAL PROCESSING-IMAGE COMMUNICATION
ISSN	0923-5965
	2023-11-01
卷号	119 页码:11
通讯作者	Fang, Yuming(fa0001ng@e.ntu.edu.sg)
摘要	Light Field (LF) imaging can record both the intensities and directions of light rays in a single exposure, which has received extensive attentions. However, the limited angular resolution becomes the primary bottleneck for the wide-spread applications of LF imaging. To this end, this paper proposes a novel optical flow-assisted multi-level fusion network for LF angular reconstruction. In our method, we propose to infer the multi-angular optical flows to explore long-range dependency of LF sub-aperture images (SAIs) for high-quality angular reconstruction. By aligning the SAIs in multi-angular directions, the geometric consistency of reconstructed dense LF can be preserved. Moreover, a multi-level fusion framework for LF angular reconstruction is introduced, which consists of two stages, namely texture-optical flow feature fusion and parallax structure-information fusion. The former firstly extracts the texture and optical flow features from the reconstructed coarse LF and then fuses these two features by using the proposed texture-optical flow fusion-block. The latter further blends the LF parallax structure information with the fused texture and optical flow features using the proposed parallax structure-information fusion network. Comprehensive experiments on both real-world and synthetic LF scenes demonstrate the superiority of the proposed method for reconstructing high-quality dense LF. Moreover, practical application on depth estimation also validates that our method can recover more texture details, particularly for some occlusion regions.
关键词	Light field image Angular reconstruction Optical flow Multi-level fusion network
DOI	10.1016/j.image.2023.117031
关键词[WOS]	SUPERRESOLUTION
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[62171002] ; National Natural Science Foundation of China[62132006] ; Shenzhen Municipal Science and Technology Innovation Council[2021Szvup051] ; STCSM[SKLSFO2021-05] ; Anhui Provincial Key Laboratory of Network and Information Security[AHNIS2023002] ; University Discipline Top Talent Program of Anhui[gxbjZD2022034] ; Anhui Outstanding Youth Fund by Colleges and Universities[2022AH030106]
项目资助者	National Natural Science Foundation of China ; Shenzhen Municipal Science and Technology Innovation Council ; STCSM ; Anhui Provincial Key Laboratory of Network and Information Security ; University Discipline Top Talent Program of Anhui ; Anhui Outstanding Youth Fund by Colleges and Universities
WOS研究方向	Engineering
WOS类目	Engineering, Electrical & Electronic
WOS记录号	WOS:001069612700001
出版者	ELSEVIER
引用统计
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/53129
专题	多模态人工智能系统全国重点实验室
通讯作者	Fang, Yuming
作者单位	1.Anqing Normal Univ, Sch Comp & Informat, Anqing 246000, Peoples R China 2.Jiangxi Univ Finance & Econ, Sch Informat Management, Nanchang 330000, Jiangxi, Peoples R China 3.Chinese Acad Sci CASIA, Natl Lab Pattern Recognit NLPR, Inst Automat, Beijing 100190, Peoples R China 4.Anhui Normal Univ, Natl Anhui Prov Key Lab Network & Informat Secur, Wuhu 240002, Peoples R China
推荐引用方式 GB/T 7714	Liu, Deyang,Mao, Yifan,Huang, Yan,et al. Optical flow-assisted multi-level fusion network for Light Field image angular reconstruction[J]. SIGNAL PROCESSING-IMAGE COMMUNICATION,2023,119:11.
APA	Liu, Deyang,Mao, Yifan,Huang, Yan,Cao, Liqun,Wang, Yuanzhi,&Fang, Yuming.(2023).Optical flow-assisted multi-level fusion network for Light Field image angular reconstruction.SIGNAL PROCESSING-IMAGE COMMUNICATION,119,11.
MLA	Liu, Deyang,et al."Optical flow-assisted multi-level fusion network for Light Field image angular reconstruction".SIGNAL PROCESSING-IMAGE COMMUNICATION 119(2023):11.