CASIA OpenIR  > 数字内容技术与服务研究中心  > 版权智能与文化计算
Inductive Zero-Shot Image Annotation via Embedding Graph
Wang, Fangxin1,2; Liu, Jie1; Zhang, Shuwu1,3; Zhang, Guixuan1; Li, Yuejun1,2; Yuan, Fei1
Source PublicationIEEE ACCESS
ISSN2169-3536
2019
Volume7Pages:107816-107830
Corresponding AuthorLiu, Jie(jie.liu@ia.ac.cn)
AbstractConventional image annotation systems can only handle those images having labels within the exist library, but cannot recognize those novel labels. In order to learn new concepts, one has to gather large amount of labeled images and train the model from scratch. More importantly, it can come with a high price to collect those labeled images. For these reasons, we put forward a zero-shot image annotation model, to reduce the demand for the images with novel labels. In this paper, we focus on the two big challenges of zero-shot image annotation: polysemous words and a strong bias in the generalized zero-shot setting. For the first problem, instead of training on large corpus datasets as previous methods, we propose to adopt Node2Vec to obtain contextualized word embeddings, which can easily produce word vectors of the polysemous words. For the second problem, we alleviate the strong bias in two ways: on one hand, we utilize a model based on graph convolutional network (GCN) to make target images involved in the training process; on the other hand, we put forward a novel semantic coherent (SC) loss to capture the semantic relations of the source and target labels. The extensive experiments on NUSWIDE, COCO, IAPR TC-12, and Core15k datasets show the superiority of the proposed model and the annotation performance get improved by 4%-6% comparing with state-of-the-art methods.
KeywordContextualized word embeddings graph convolutional network image annotation Node2Vec zero-shot
DOI10.1109/ACCESS.2019.2925383
Indexed BySCI
Language英语
Funding ProjectNational Key R&D Program of China[2018YFC0809200] ; National Key Research and Development Plan[61602480] ; National Key R&D Program of China[2018YFC0809200] ; National Key Research and Development Plan[61602480]
Funding OrganizationNational Key R&D Program of China ; National Key Research and Development Plan
WOS Research AreaComputer Science ; Engineering ; Telecommunications
WOS SubjectComputer Science, Information Systems ; Engineering, Electrical & Electronic ; Telecommunications
WOS IDWOS:000481980800008
PublisherIEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Citation statistics
Cited Times:1[WOS]   [WOS Record]     [Related Records in WOS]
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/26113
Collection数字内容技术与服务研究中心_版权智能与文化计算
Corresponding AuthorLiu, Jie
Affiliation1.Chinese Acad Sci, Inst Automat, Beijing, Peoples R China
2.Univ Chinese Acad Sci, Beijing, Peoples R China
3.Beijing Film Acad, AICFVE, Beijing, Peoples R China
First Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Corresponding Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Wang, Fangxin,Liu, Jie,Zhang, Shuwu,et al. Inductive Zero-Shot Image Annotation via Embedding Graph[J]. IEEE ACCESS,2019,7:107816-107830.
APA Wang, Fangxin,Liu, Jie,Zhang, Shuwu,Zhang, Guixuan,Li, Yuejun,&Yuan, Fei.(2019).Inductive Zero-Shot Image Annotation via Embedding Graph.IEEE ACCESS,7,107816-107830.
MLA Wang, Fangxin,et al."Inductive Zero-Shot Image Annotation via Embedding Graph".IEEE ACCESS 7(2019):107816-107830.
Files in This Item: Download All
File Name/Size DocType Version Access License
FINAL Article.pdf(1472KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Wang, Fangxin]'s Articles
[Liu, Jie]'s Articles
[Zhang, Shuwu]'s Articles
Baidu academic
Similar articles in Baidu academic
[Wang, Fangxin]'s Articles
[Liu, Jie]'s Articles
[Zhang, Shuwu]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Wang, Fangxin]'s Articles
[Liu, Jie]'s Articles
[Zhang, Shuwu]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: FINAL Article.pdf
Format: Adobe PDF
This file does not support browsing at this time
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.