CASIA OpenIR  > 类脑智能研究中心
Image Caption Generation with Part of Speech Guidance
Xinwei He; Baoguang Shi; Xiang Bai; Gui-Song Xia; Zhaoxiang Zhang; Weisheng Dong
Source PublicationPattern Recognition Letters
AbstractAs a fundamental problem in image understanding, image caption generation has attracted much attention from both computer vision and natural language processing communities. In this paper, we focus on how to exploit the structure information of a natural sentence, which is used to describe the content of an image. We discover that the Part of Speech (PoS) tags of a sentence, are very effective cues for guiding the Long Short-Term Memory (LSTM) based word generator. More specifically, given a sentence, the PoS tag of each word is utilized to determine whether it is essential to input image representation into the word generator. Benefiting from such a strategy, our model can closely connect the visual attributes of an image to the word concepts in the natural language space. Experimental results on the most popular benchmark datasets, e.g., Flickr30k and MS COCO, consistently demonstrate that our method can significantly enhance the performance of a standard image caption generation model, and achieve the conpetitive results.
KeywordImage Caption Generation Part-of-speech Tags Long Short-term Memory Visual Attributes
WOS IDWOS:000458876700028
Citation statistics
Document Type期刊论文
Recommended Citation
GB/T 7714
Xinwei He,Baoguang Shi,Xiang Bai,et al. Image Caption Generation with Part of Speech Guidance[J]. Pattern Recognition Letters,2017(1):1-9.
APA Xinwei He,Baoguang Shi,Xiang Bai,Gui-Song Xia,Zhaoxiang Zhang,&Weisheng Dong.(2017).Image Caption Generation with Part of Speech Guidance.Pattern Recognition Letters(1),1-9.
MLA Xinwei He,et al."Image Caption Generation with Part of Speech Guidance".Pattern Recognition Letters .1(2017):1-9.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Xinwei He]'s Articles
[Baoguang Shi]'s Articles
[Xiang Bai]'s Articles
Baidu academic
Similar articles in Baidu academic
[Xinwei He]'s Articles
[Baoguang Shi]'s Articles
[Xiang Bai]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Xinwei He]'s Articles
[Baoguang Shi]'s Articles
[Xiang Bai]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.