Domain-Oriented Semantic Embedding for Zero-Shot Learning

doi:10.1109/TMM.2020.3033124

	Domain-Oriented Semantic Embedding for Zero-Shot Learning
	Min, Shaobo 1; Yao, Hantao2 ; Xie, Hongtao 1; Zha, Zheng-Jun 1; Zhang, Yongdong 1
发表期刊	IEEE TRANSACTIONS ON MULTIMEDIA
ISSN	1520-9210
	2021
卷号	23 页码:3919-3930
通讯作者	Xie, Hongtao(htxie@ustc.edu.cn) ; Zhang, Yongdong(zhyd73@ustc.edu.cn)
摘要	Zero-Shot Learning (ZSL) targets to recognize images from new classes. Existing methods focus on learning a projection function to associate the visual features and category descriptions in the seen domain, which is directly transferred to the unseen domain. However, due to the inherent domain shift, a single shared projection cannot fully capture the domain difference and similarity, thereby making the unseen samples tend to be recognized as seen categories. In this paper, we propose a novel Domain-Oriented Semantic Embedding (DOSE) network that learns specific projections for different domains to better capture the domain characteristics for unbiased ZSL. Besides a domain-shared projection, DOSE learns two auxiliary domain-specific sub-projections to model the semantic-visual association in respective seen and unseen domains. Specifically, the domain-specific projections are learned in a cycle consistency way to capture domain characteristics, and a domain division constraint is developed to penalize the margin between two domain embeddings. Furthermore, to boost semantic-visual association, a semantic-visual dual attention module is designed to automatically remove trivial information in both visual and semantic embeddings under a co-guidance learning manner. Experiments on four public benchmarks prove that the proposed DOSE is robust to the domain shift problem in ZSL and obtains an averaged 5.6% improvement in terms of harmonic mean.
关键词	Semantics Visualization Image recognition Image reconstruction Training Gallium nitride Search problems Zero-shot learning multi-modality embedding recognition
DOI	10.1109/TMM.2020.3033124
收录类别	SCI
语种	英语
资助项目	National Key Research, and Development Program of China[2017YFC0820600] ; National Nature Science Foundation of China[61525206] ; National Nature Science Foundation of China[62022076] ; National Nature Science Foundation of China[U1936210] ; National Postdoctoral Programme for Innovative Talents[BX20180358] ; Youth Innovation Promotion Association Chinese Academy of Sciences[2017209]
项目资助者	National Key Research, and Development Program of China ; National Nature Science Foundation of China ; National Postdoctoral Programme for Innovative Talents ; Youth Innovation Promotion Association Chinese Academy of Sciences
WOS研究方向	Computer Science ; Telecommunications
WOS类目	Computer Science, Information Systems ; Computer Science, Software Engineering ; Telecommunications
WOS记录号	WOS:000709093100038
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：6[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/46307
专题	多模态人工智能系统全国重点实验室_多媒体计算
通讯作者	Xie, Hongtao; Zhang, Yongdong
作者单位	1.Univ Sci & Technol China, Natl Engn Lab Brain Inspired Intelligence Technol, Hefei 230026, Peoples R China 2.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100864, Peoples R China
推荐引用方式 GB/T 7714	Min, Shaobo,Yao, Hantao,Xie, Hongtao,et al. Domain-Oriented Semantic Embedding for Zero-Shot Learning[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2021,23:3919-3930.
APA	Min, Shaobo,Yao, Hantao,Xie, Hongtao,Zha, Zheng-Jun,&Zhang, Yongdong.(2021).Domain-Oriented Semantic Embedding for Zero-Shot Learning.IEEE TRANSACTIONS ON MULTIMEDIA,23,3919-3930.
MLA	Min, Shaobo,et al."Domain-Oriented Semantic Embedding for Zero-Shot Learning".IEEE TRANSACTIONS ON MULTIMEDIA 23(2021):3919-3930.