Inductive Zero-Shot Image Annotation via Embedding Graph | |
Wang, Fangxin1,2; Liu, Jie1; Zhang, Shuwu1,3; Zhang, Guixuan1; Li, Yuejun1,2; Yuan, Fei1 | |
发表期刊 | IEEE ACCESS |
ISSN | 2169-3536 |
2019 | |
卷号 | 7页码:107816-107830 |
通讯作者 | Liu, Jie(jie.liu@ia.ac.cn) |
摘要 | Conventional image annotation systems can only handle those images having labels within the exist library, but cannot recognize those novel labels. In order to learn new concepts, one has to gather large amount of labeled images and train the model from scratch. More importantly, it can come with a high price to collect those labeled images. For these reasons, we put forward a zero-shot image annotation model, to reduce the demand for the images with novel labels. In this paper, we focus on the two big challenges of zero-shot image annotation: polysemous words and a strong bias in the generalized zero-shot setting. For the first problem, instead of training on large corpus datasets as previous methods, we propose to adopt Node2Vec to obtain contextualized word embeddings, which can easily produce word vectors of the polysemous words. For the second problem, we alleviate the strong bias in two ways: on one hand, we utilize a model based on graph convolutional network (GCN) to make target images involved in the training process; on the other hand, we put forward a novel semantic coherent (SC) loss to capture the semantic relations of the source and target labels. The extensive experiments on NUSWIDE, COCO, IAPR TC-12, and Core15k datasets show the superiority of the proposed model and the annotation performance get improved by 4%-6% comparing with state-of-the-art methods. |
关键词 | Contextualized word embeddings graph convolutional network image annotation Node2Vec zero-shot |
DOI | 10.1109/ACCESS.2019.2925383 |
收录类别 | SCI |
语种 | 英语 |
资助项目 | National Key R&D Program of China[2018YFC0809200] ; National Key Research and Development Plan[61602480] ; National Key R&D Program of China[2018YFC0809200] ; National Key Research and Development Plan[61602480] |
项目资助者 | National Key R&D Program of China ; National Key Research and Development Plan |
WOS研究方向 | Computer Science ; Engineering ; Telecommunications |
WOS类目 | Computer Science, Information Systems ; Engineering, Electrical & Electronic ; Telecommunications |
WOS记录号 | WOS:000481980800008 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
七大方向——子方向分类 | 多模态智能 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/26113 |
专题 | 数字内容技术与服务研究中心_版权智能与文化计算 |
通讯作者 | Liu, Jie |
作者单位 | 1.Chinese Acad Sci, Inst Automat, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Beijing, Peoples R China 3.Beijing Film Acad, AICFVE, Beijing, Peoples R China |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Wang, Fangxin,Liu, Jie,Zhang, Shuwu,et al. Inductive Zero-Shot Image Annotation via Embedding Graph[J]. IEEE ACCESS,2019,7:107816-107830. |
APA | Wang, Fangxin,Liu, Jie,Zhang, Shuwu,Zhang, Guixuan,Li, Yuejun,&Yuan, Fei.(2019).Inductive Zero-Shot Image Annotation via Embedding Graph.IEEE ACCESS,7,107816-107830. |
MLA | Wang, Fangxin,et al."Inductive Zero-Shot Image Annotation via Embedding Graph".IEEE ACCESS 7(2019):107816-107830. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
FINAL Article.pdf(1472KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论