Learning stratified 3D reconstruction

doi:10.1007/s11432-017-9234-7

	Learning stratified 3D reconstruction
	Dong, Qiulei1,3,4 ; Shu, Mao1 ; Cui, Hainan1 ; Xu, Huarong 1,2; Hu, Zhanyi1,3,4
发表期刊	SCIENCE CHINA-INFORMATION SCIENCES
	2018-02-01
卷号	61 期号:2 页码:1-16
文章类型	Article
摘要	Stratified 3D reconstruction, or a layer-by-layer 3D reconstruction upgraded from projective to affine, then to the final metric reconstruction, is a well-known 3D reconstruction method in computer vision. It is also a key supporting technology for various well-known applications, such as streetview, smart3D, oblique photogrammetry. Generally speaking, the existing computer vision methods in the literature can be roughly classified into either the geometry-based approaches for spatial vision or the learning-based approaches for object vision. Although deep learning has demonstrated tremendous success in object vision in recent years, learning 3D scene reconstruction from multiple images is still rare, even not existent, except for those on depth learning from single images. This study is to explore the feasibility of learning the stratified 3D reconstruction from putative point correspondences across images, and to assess whether it could also be as robust to matching outliers as the traditional geometry-based methods do. In this study, a special parsimonious neural network is designed for the learning. Our results show that it is indeed possible to learn a stratified 3D reconstruction from noisy image point correspondences, and the learnt reconstruction results appear satisfactory although they are still not on a par with the state-of-the-arts in the structure-from-motion community due to largely its lack of an explicit robust outlier detector such as random sample consensus (RANSAC). To the best of our knowledge, our study is the first attempt in the literature to learn 3D scene reconstruction from multiple images. Our results also show that how to implicitly or explicitly integrate an outlier detector in learning methods is a key problem to solve in order to learn comparable 3D scene structures to those by the current geometry-based state-of-the-arts. Otherwise any significant advancement of learning 3D structures from multiple images seems difficult, if not impossible. Besides, we even speculate that deep learning might be, in nature, not suitable for learning 3D structure from multiple images, or more generally, for solving spatial vision problems.
关键词	Stratified 3d Reconstruction Learning Deep Neural Networks Outlier Detector Spatial Vision
WOS标题词	Science & Technology ; Technology
DOI	10.1007/s11432-017-9234-7
收录类别	SCI
语种	英语
项目资助者	National Natural Science Foundation of China(61333015 ; 61375042 ; 61421004 ; 61573359 ; 61772444)
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Information Systems ; Engineering, Electrical & Electronic
WOS记录号	WOS:000423635200003
引用统计	被引频次：8[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/21946
专题	多模态人工智能系统全国重点实验室_机器人视觉
作者单位	1.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 2.Xiamen Inst Technol, Dept Comp Sci & Technol, Xiamen 361024, Peoples R China 3.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 4.Chinese Acad Sci, Ctr Excellence Brain Sci & Intelligence Technol, Beijing 100049, Peoples R China
第一作者单位	中国科学院自动化研究所
推荐引用方式 GB/T 7714	Dong, Qiulei,Shu, Mao,Cui, Hainan,et al. Learning stratified 3D reconstruction[J]. SCIENCE CHINA-INFORMATION SCIENCES,2018,61(2):1-16.
APA	Dong, Qiulei,Shu, Mao,Cui, Hainan,Xu, Huarong,&Hu, Zhanyi.(2018).Learning stratified 3D reconstruction.SCIENCE CHINA-INFORMATION SCIENCES,61(2),1-16.
MLA	Dong, Qiulei,et al."Learning stratified 3D reconstruction".SCIENCE CHINA-INFORMATION SCIENCES 61.2(2018):1-16.

条目包含的文件		下载所有文件
文件名称/大小	文献类型	版本类型	开放类型	使用许可
Dong2017_Article_Lea（3578KB）	期刊论文	作者接受稿	开放获取	CC BY-NC-SA	浏览下载