CASIA OpenIR  > 模式识别国家重点实验室  > 自然语言处理
Multi-modal Sentence Summarization with Modality Attention and Image Filtering
Haoran Li1,2; Junnan Zhu1,2; Tianshang Liu1,2; Jiajun Zhang1,2; Chengqing Zong1,2,3
2018
会议名称Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
会议日期2018
会议地点Sweden
出版者长文
摘要

In this paper, we introduce a multi-modal sentence summarization task that produces a short summary from a pair of sentence and image. This task is more challenging than sentence summarization. It not only needs to effectively incorporate visual features into standard text summarization framework, but also requires to avoid noise of image. To this end, we propose a modality-based attention mechanism to pay different attention to image patches and text units, and we design image filters to selectively
use visual information to enhance the semantics of the input sentence. We construct a multimodal sentence summarization dataset and extensive
experiments on this dataset demonstrate that our models significantly outperform conventional models which only employ text as input. Further
analyses suggest that sentence summarization task can benefit from visually grounded representations from a variety of aspects.

文献类型会议论文
条目标识符http://ir.ia.ac.cn/handle/173211/23108
专题模式识别国家重点实验室_自然语言处理
作者单位1.National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences
2.University of Chinese Academy of Sciences
3.CAS Center for Excellence in Brain Science and Intelligence Technology
推荐引用方式
GB/T 7714
Haoran Li,Junnan Zhu,Tianshang Liu,et al. Multi-modal Sentence Summarization with Modality Attention and Image Filtering[C]:长文,2018.
条目包含的文件 下载所有文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
3IJCAI.pdf(352KB)会议论文 开放获取CC BY-NC-SA浏览 下载
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[Haoran Li]的文章
[Junnan Zhu]的文章
[Tianshang Liu]的文章
百度学术
百度学术中相似的文章
[Haoran Li]的文章
[Junnan Zhu]的文章
[Tianshang Liu]的文章
必应学术
必应学术中相似的文章
[Haoran Li]的文章
[Junnan Zhu]的文章
[Tianshang Liu]的文章
相关权益政策
暂无数据
收藏/分享
文件名: 3IJCAI.pdf
格式: Adobe PDF
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。