CASIA OpenIR  > 模式识别国家重点实验室  > 自然语言处理
MSMO: Multimodal Summarization with Multimodal Output
Zhu, Junnan; Li, Haoran; Liu, Tianshang; Zhou, Yu; Zhang, Jiajun; Zong, Chengqing
2018
会议名称EMNLP
会议日期2018-11
会议地点Brussels, Belgium
摘要

Multimodal summarization has drawn much attention due to the rapid growth of multimedia data. The output of the current multimodal summarization systems is usually represented in texts. However, we have found through experiments that multimodal output can significantly improve user satisfaction for informativeness of summaries. In this paper, we propose a novel task, multimodal summarization with multimodal output (MSMO). To handle this task, we first collect a large-scale dataset for MSMO research. We then propose a multimodal attention model to jointly generate text and select the most relevant image from the multimodal input. Finally, to evaluate multimodal outputs, we construct a novel multimodal automatic evaluation (MMAE) method which considers both intramodality salience and intermodality relevance. The experimental results show the effectiveness of MMAE.

文献类型会议论文
条目标识符http://ir.ia.ac.cn/handle/173211/23200
专题模式识别国家重点实验室_自然语言处理
作者单位中国科学院自动化研究所
第一作者单位中国科学院自动化研究所
推荐引用方式
GB/T 7714
Zhu, Junnan,Li, Haoran,Liu, Tianshang,et al. MSMO: Multimodal Summarization with Multimodal Output[C],2018.
条目包含的文件
条目无相关文件。
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[Zhu, Junnan]的文章
[Li, Haoran]的文章
[Liu, Tianshang]的文章
百度学术
百度学术中相似的文章
[Zhu, Junnan]的文章
[Li, Haoran]的文章
[Liu, Tianshang]的文章
必应学术
必应学术中相似的文章
[Zhu, Junnan]的文章
[Li, Haoran]的文章
[Liu, Tianshang]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。