CASIA OpenIR
Collaborative Deconvolutional Neural Networks for Joint Depth Estimation and Semantic Segmentation
Liu, Jing1; Wang, Yuhang1,2; Li, Yong1,2; Fu, Jun1,2; Li, Jiangyun3; Lu, Hanqing1
Source PublicationIEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
ISSN2162-237X
2018-11-01
Volume29Issue:11Pages:5655-5666
Corresponding AuthorLiu, Jing(jliu@nlpr.ia.ac.cn)
AbstractSemantic segmentation and single-view depth estimation are two fundamental problems in computer vision. They exploit the semantic and geometric properties of images, respectively, and are thus complementary in scene understanding. In this paper, we propose a collaborative deconvolutional neural network (C-DCNN) to jointly model these two problems for mutual promotion. The C-DCNN consists of two DCNNs, of which each is for one task. The DCNNs provide a finer resolution reconstruction method and are pretrained with hierarchical supervision. The feature maps from these two DCNNs are integrated via a pointwise bilinear layer, which fuses the semantic and depth information and produces higher order features. Then, the integrated features are fed into two sibling classification layers to simultaneously learn for semantic segmentation and depth estimation. In this way, we combine the semantic and depth features in a unified deep network and jointly train them to benefit each other. Specifically, during network training, we process depth estimation as a classification problem where a soft mapping strategy is proposed to map the continuous depth values into discrete probability distributions and the cross entropy loss is used. Besides, a fully connected conditional random field is also used as postprocessing to further improve the performance of semantic segmentation, where the proximity relations of pixels on position, intensity, and depth are jointly considered. We evaluate our approach on two challenging benchmarks: NYU Depth V2 and SUN RGB-D. It is demonstrated that our approach effectively utilizes these two kinds of information and achieves state-of-the-art results on both the semantic segmentation and depth estimation tasks.
KeywordDeconvolutional neural network (DCNN) depth estimation fully connected conditional random field (CRF) pointwise bilinear layer semantic segmentation soft mapping strategy
DOI10.1109/TNNLS.2017.2787781
WOS KeywordRECOGNITION
Indexed BySCI
Language英语
Funding ProjectNational Natural Science Foundation of China[61332016] ; National Natural Science Foundation of China[61472422] ; Fundamental Research Funds for Central Universities[FRFBD-16-005A]
Funding OrganizationNational Natural Science Foundation of China ; Fundamental Research Funds for Central Universities
WOS Research AreaComputer Science ; Engineering
WOS SubjectComputer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS IDWOS:000447832200038
PublisherIEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Citation statistics
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/28102
Collection中国科学院自动化研究所
Corresponding AuthorLiu, Jing
Affiliation1.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China
2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
3.Univ Sci & Technol, Beijing 100083, Peoples R China
First Author AffilicationChinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China
Corresponding Author AffilicationChinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China
Recommended Citation
GB/T 7714
Liu, Jing,Wang, Yuhang,Li, Yong,et al. Collaborative Deconvolutional Neural Networks for Joint Depth Estimation and Semantic Segmentation[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2018,29(11):5655-5666.
APA Liu, Jing,Wang, Yuhang,Li, Yong,Fu, Jun,Li, Jiangyun,&Lu, Hanqing.(2018).Collaborative Deconvolutional Neural Networks for Joint Depth Estimation and Semantic Segmentation.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,29(11),5655-5666.
MLA Liu, Jing,et al."Collaborative Deconvolutional Neural Networks for Joint Depth Estimation and Semantic Segmentation".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 29.11(2018):5655-5666.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Liu, Jing]'s Articles
[Wang, Yuhang]'s Articles
[Li, Yong]'s Articles
Baidu academic
Similar articles in Baidu academic
[Liu, Jing]'s Articles
[Wang, Yuhang]'s Articles
[Li, Yong]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Liu, Jing]'s Articles
[Wang, Yuhang]'s Articles
[Li, Yong]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.