Adaptive Attention Annotation Model: Optimizing the Prediction Path Through Dependency Fusion
Wang Fangxin(王方心)1,2; Liu Jie2; Zhang Shuwu2,3; Zhang Guixuan2; Zheng Yang2; Li Xiaoqian1,2; Liang Wei2; Li Yuejun1,2
Source PublicationKSII Transactions on Internet and Information Systems
2019-09
Issue9Pages:4665-4683
Abstract

Previous methods build image annotation model by leveraging three basic dependencies: relations between image and label (image/label), between images (image/image) and between labels (label/label). Even though plenty of researches show that multiple dependencies can work jointly to improve annotation performance, different dependencies actually do not "work jointly" in their diagram, whose performance is largely depending on the result predicted by image/label section. To address this problem, we propose the adaptive attention annotation model (AAAM) to associate these dependencies with the prediction path, which is composed of a series of labels (tags) in the order they are detected. In particular, we optimize the prediction path by detecting the relevant labels from the easy-to-detect to the hard-to-detect, which are found using Binary Cross-Entropy (BCE) and Triplet Margin (TM) losses, respectively. Besides, in order to capture the inforamtion of each label, instead of explicitly extracting regional featutres, we propose the self-attention machanism to implicitly enhance the relevant region and restrain those irrelevant. To validate the effective of the model, we conduct experiments on three well-known public datasets, COCO 2014, IAPR TC-12 and NUSWIDE, and achieve better performance than the state-of-the-art methods.

KeywordImage Annotation Multiple Dependencies Self-attention Prediction Path Triplet Margin Loss
Language英语
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/26114
Collection数字内容技术与服务研究中心_新媒体服务与管理技术
Corresponding AuthorLiu Jie
Affiliation1.中国科学院大学
2.中国科学院自动化研究所
3.北京电影学院,AICFVE
First Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Corresponding Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Wang Fangxin,Liu Jie,Zhang Shuwu,et al. Adaptive Attention Annotation Model: Optimizing the Prediction Path Through Dependency Fusion[J]. KSII Transactions on Internet and Information Systems,2019(9):4665-4683.
APA Wang Fangxin.,Liu Jie.,Zhang Shuwu.,Zhang Guixuan.,Zheng Yang.,...&Li Yuejun.(2019).Adaptive Attention Annotation Model: Optimizing the Prediction Path Through Dependency Fusion.KSII Transactions on Internet and Information Systems(9),4665-4683.
MLA Wang Fangxin,et al."Adaptive Attention Annotation Model: Optimizing the Prediction Path Through Dependency Fusion".KSII Transactions on Internet and Information Systems .9(2019):4665-4683.
Files in This Item:
File Name/Size DocType Version Access License
TIIS Vol 13, No 9-19(1061KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Application Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Wang Fangxin(王方心)]'s Articles
[Liu Jie]'s Articles
[Zhang Shuwu]'s Articles
Baidu academic
Similar articles in Baidu academic
[Wang Fangxin(王方心)]'s Articles
[Liu Jie]'s Articles
[Zhang Shuwu]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Wang Fangxin(王方心)]'s Articles
[Liu Jie]'s Articles
[Zhang Shuwu]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: TIIS Vol 13, No 9-19.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.