Inductive Zero-Shot Image Annotation via Embedding Graph
Wang Fangxin(王方心)1,2; Liu Jie2; Zhang Shuwu2,3; Zhang Guixuan2; Li Yuejun1,2; Yuan Fei2
Source PublicationIEEE Access
2019-06
Volume7Issue:0Pages:107816-107830
Abstract

Conventional image annotation systems can only handle those images having labels within the exist library, but cannot recognize those novel labels. In order to learn new concepts, one has to gather large amount of labeled images and train the model from scratch. More importantly, it can come with a high price to collect those labeled images. For these reasons, we put forward a zero-shot image annotation model, to reduce the demand for the images with novel labels. In this paper, we focus on the two big challenges of zero-shot image annotation: polysemous words and a strong bias in the generalized zero-shot setting. For the first problem, instead of training on large corpus datasets as previous methods, we propose to adopt Node2Vec to obtain contextualized word embeddings, which can easily produce word vectors of the polysemous words. For the second problem, we alleviate the strong bias in two ways: on one hand, we utilize a model based on graph convolutional network (GCN) to make target images involved in the training process; on the other hand, we put forward a novel semantic coherent (SC) loss to capture the semantic relations of the source and target labels. The extensive experiments on NUSWIDE, COCO, IAPR TC-12, and Corel5k datasets show the superiority of the proposed model and the annotation performance get improved by 4%–6% comparing with state-of-the-art methods.

KeywordContextualized Word Embeddings Graph Convolutional Network Image Annotation Node2vec Zero-shot
Language英语
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/26113
Collection数字内容技术与服务研究中心_新媒体服务与管理技术
Corresponding AuthorLiu Jie
Affiliation1.中国科学院大学
2.中国科学院自动化研究所
3.北京电影学院,AICFVE
First Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Corresponding Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Wang Fangxin,Liu Jie,Zhang Shuwu,et al. Inductive Zero-Shot Image Annotation via Embedding Graph[J]. IEEE Access,2019,7(0):107816-107830.
APA Wang Fangxin,Liu Jie,Zhang Shuwu,Zhang Guixuan,Li Yuejun,&Yuan Fei.(2019).Inductive Zero-Shot Image Annotation via Embedding Graph.IEEE Access,7(0),107816-107830.
MLA Wang Fangxin,et al."Inductive Zero-Shot Image Annotation via Embedding Graph".IEEE Access 7.0(2019):107816-107830.
Files in This Item: Download All
File Name/Size DocType Version Access License
FINAL Article.pdf(1472KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Wang Fangxin(王方心)]'s Articles
[Liu Jie]'s Articles
[Zhang Shuwu]'s Articles
Baidu academic
Similar articles in Baidu academic
[Wang Fangxin(王方心)]'s Articles
[Liu Jie]'s Articles
[Zhang Shuwu]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Wang Fangxin(王方心)]'s Articles
[Liu Jie]'s Articles
[Zhang Shuwu]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: FINAL Article.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.