CASIA OpenIR  > 智能感知与计算
Unconstrained Multimodal Multi-Label Learning
Huang, Yan1; Wang, Wei1; Wang, Liang1,2
AbstractMultimodal learning has been mostly studied by assuming that multiple label assignments are independent of each other and all the modalities are available. In this paper, we consider a more general problem where the labels contain dependency relationships and some modalities are likely to be missing. To this end, we propose a multi-label conditional restricted Boltzmann machine (ML-CRBM), which handles modality completion, fusion, and multi-label prediction in a unified framework. The proposed model is able to generate missing modalities based on observed ones, by explicitly modelling and sampling their conditional distributions. After that, it can discriminatively fuse multiple modalities to obtain shared representations under the supervision of class labels. To consider the co-occurrence of the labels, the proposed model formulates the multi-label prediction as a max-margin-based multi-task learning problem. Model parameters can be jointly learned by seeking a balance between being generative for modality generation and being discriminative for label prediction. We perform a series of experiments in terms of classification, visualization, and retrieval, and the experimental results clearly demonstrate the effectiveness of our method.
KeywordMulti-label Learning Multi-task Learning Multimodal Learning Restricted Boltzmann Machine
WOS HeadingsScience & Technology ; Technology
Indexed BySCI
Funding OrganizationNational Natural Science Foundation of China(61175003 ; National Basic Research Program of China(2012CB316300) ; 61202328 ; 61572504 ; 61420106015)
WOS Research AreaComputer Science ; Telecommunications
WOS SubjectComputer Science, Information Systems ; Computer Science, Software Engineering ; Telecommunications
WOS IDWOS:000364102400006
Citation statistics
Cited Times:30[WOS]   [WOS Record]     [Related Records in WOS]
Document Type期刊论文
Affiliation1.Chinese Acad Sci CASIA, Inst Automat, Natl Lab Pattern Recognit, Ctr Res Intelligent Percept & Comp, Beijing 100190, Peoples R China
2.CASIA, Inst Automat, CAS Ctr Excellence Brain Sci & Intelligence Techn, Beijing 100190, Peoples R China
First Author AffilicationChinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China
Recommended Citation
GB/T 7714
Huang, Yan,Wang, Wei,Wang, Liang. Unconstrained Multimodal Multi-Label Learning[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2015,17(11):1923-1935.
APA Huang, Yan,Wang, Wei,&Wang, Liang.(2015).Unconstrained Multimodal Multi-Label Learning.IEEE TRANSACTIONS ON MULTIMEDIA,17(11),1923-1935.
MLA Huang, Yan,et al."Unconstrained Multimodal Multi-Label Learning".IEEE TRANSACTIONS ON MULTIMEDIA 17.11(2015):1923-1935.
Files in This Item: Download All
File Name/Size DocType Version Access License
UMMLL.pdf(3011KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Huang, Yan]'s Articles
[Wang, Wei]'s Articles
[Wang, Liang]'s Articles
Baidu academic
Similar articles in Baidu academic
[Huang, Yan]'s Articles
[Wang, Wei]'s Articles
[Wang, Liang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Huang, Yan]'s Articles
[Wang, Wei]'s Articles
[Wang, Liang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: UMMLL.pdf
Format: Adobe PDF
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.