CASIA OpenIR  > 模式识别国家重点实验室  > 机器人视觉
Learning stratified 3D reconstruction
Dong, Qiulei1,3,4; Shu, Mao1; Cui, Hainan1; Xu, Huarong1,2; Hu, Zhanyi1,3,4
AbstractStratified 3D reconstruction, or a layer-by-layer 3D reconstruction upgraded from projective to affine, then to the final metric reconstruction, is a well-known 3D reconstruction method in computer vision. It is also a key supporting technology for various well-known applications, such as streetview, smart3D, oblique photogrammetry. Generally speaking, the existing computer vision methods in the literature can be roughly classified into either the geometry-based approaches for spatial vision or the learning-based approaches for object vision. Although deep learning has demonstrated tremendous success in object vision in recent years, learning 3D scene reconstruction from multiple images is still rare, even not existent, except for those on depth learning from single images. This study is to explore the feasibility of learning the stratified 3D reconstruction from putative point correspondences across images, and to assess whether it could also be as robust to matching outliers as the traditional geometry-based methods do. In this study, a special parsimonious neural network is designed for the learning. Our results show that it is indeed possible to learn a stratified 3D reconstruction from noisy image point correspondences, and the learnt reconstruction results appear satisfactory although they are still not on a par with the state-of-the-arts in the structure-from-motion community due to largely its lack of an explicit robust outlier detector such as random sample consensus (RANSAC). To the best of our knowledge, our study is the first attempt in the literature to learn 3D scene reconstruction from multiple images. Our results also show that how to implicitly or explicitly integrate an outlier detector in learning methods is a key problem to solve in order to learn comparable 3D scene structures to those by the current geometry-based state-of-the-arts. Otherwise any significant advancement of learning 3D structures from multiple images seems difficult, if not impossible. Besides, we even speculate that deep learning might be, in nature, not suitable for learning 3D structure from multiple images, or more generally, for solving spatial vision problems.
KeywordStratified 3d Reconstruction Learning Deep Neural Networks Outlier Detector Spatial Vision
WOS HeadingsScience & Technology ; Technology
Indexed BySCI
Funding OrganizationNational Natural Science Foundation of China(61333015 ; 61375042 ; 61421004 ; 61573359 ; 61772444)
WOS Research AreaComputer Science ; Engineering
WOS SubjectComputer Science, Information Systems ; Engineering, Electrical & Electronic
WOS IDWOS:000423635200003
Citation statistics
Cited Times:1[WOS]   [WOS Record]     [Related Records in WOS]
Document Type期刊论文
Affiliation1.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China
2.Xiamen Inst Technol, Dept Comp Sci & Technol, Xiamen 361024, Peoples R China
3.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
4.Chinese Acad Sci, Ctr Excellence Brain Sci & Intelligence Technol, Beijing 100049, Peoples R China
Recommended Citation
GB/T 7714
Dong, Qiulei,Shu, Mao,Cui, Hainan,et al. Learning stratified 3D reconstruction[J]. SCIENCE CHINA-INFORMATION SCIENCES,2018,61(2):1-16.
APA Dong, Qiulei,Shu, Mao,Cui, Hainan,Xu, Huarong,&Hu, Zhanyi.(2018).Learning stratified 3D reconstruction.SCIENCE CHINA-INFORMATION SCIENCES,61(2),1-16.
MLA Dong, Qiulei,et al."Learning stratified 3D reconstruction".SCIENCE CHINA-INFORMATION SCIENCES 61.2(2018):1-16.
Files in This Item: Download All
File Name/Size DocType Version Access License
Dong2017_Article_Lea(3578KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Dong, Qiulei]'s Articles
[Shu, Mao]'s Articles
[Cui, Hainan]'s Articles
Baidu academic
Similar articles in Baidu academic
[Dong, Qiulei]'s Articles
[Shu, Mao]'s Articles
[Cui, Hainan]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Dong, Qiulei]'s Articles
[Shu, Mao]'s Articles
[Cui, Hainan]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: Dong2017_Article_LearningStratified3DReconstruc.pdf
Format: Adobe PDF
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.