CASIA OpenIR  > 类脑智能研究中心  > 神经计算及脑机交互
Doubly Semi-supervised Multimodal Adversarial Learning for Classification, Generation and Retrieval
Du Changde1,2; Du Changying3; He Huiguang1,2,4
2019
Conference NameIEEE International Conference on Multimedia & Expo
Conference DateJuly 8-12, 2019
Conference PlaceShanghai, China
Abstract

Learning over incomplete multi-modality data is a challenging problem with strong practical applications. Most existing multi-modal data imputation approaches have two limitations: (1) they are unable to accurately control the semantics of imputed modalities;  and (2) without a shared low-dimensional latent space, they do not scale well with multiple modalities. To overcome the limitations, we propose a novel doubly semi-supervised multi-modal learning framework (DSML) with a modality-shared latent space and modality-specific generators, encoders and classifiers. We design novel softmax-based discriminators to train all modules adversarially. As a unified framework, DSML can be applied in multi-modal semi-supervised classification, missing modality imputation and fast cross-modality retrieval tasks simultaneously. Experiments on multiple datasets demonstrate its advantages.

Document Type会议论文
Identifierhttp://ir.ia.ac.cn/handle/173211/23627
Collection类脑智能研究中心_神经计算及脑机交互
Corresponding AuthorHe Huiguang
Affiliation1.Research Center for Brain-Inspired Intelligence \& NLPR, CASIA, Beijing 100190, China
2.University of Chinese Academy of Sciences, Beijing 100049, China
3.Huawei Noah’s Ark Lab, Beijing, China
4.Center for Excellence in Brain Science and Intelligence Technology, CAS, Beijing, China
First Author AffilicationChinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China
Corresponding Author AffilicationChinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China
Recommended Citation
GB/T 7714
Du Changde,Du Changying,He Huiguang. Doubly Semi-supervised Multimodal Adversarial Learning for Classification, Generation and Retrieval[C],2019.
Files in This Item: Download All
File Name/Size DocType Version Access License
ICME2019_Doubly Semi(856KB)会议论文 开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Du Changde]'s Articles
[Du Changying]'s Articles
[He Huiguang]'s Articles
Baidu academic
Similar articles in Baidu academic
[Du Changde]'s Articles
[Du Changying]'s Articles
[He Huiguang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Du Changde]'s Articles
[Du Changying]'s Articles
[He Huiguang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: ICME2019_Doubly Semi-supervised Multimodal Adversarial Learning for Classification, Generation and Retrieval.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.