Deep Reinforcement Learning for Query-Conditioned Video Summarization
Yujia Zhang1,2; Michael Kampffmeyer3; Xiaoguang Zhao1,2; Min Tan1,2
发表期刊Applied Sciences - Basel
2019-02
卷号9期号:4页码:750-765
摘要

Query-conditioned video summarization requires to (1) find a diverse set of video shots/frames that are representative for the whole video, and that (2) the selected shots/frames are related to a given query. Thus it can be tailored to different user interests leading to a better personalized summary and differs from the generic video summarization which only focuses on video content. Our work targets this query-conditioned video summarization task, by first proposing a Mapping Network (MapNet) in order to express how related a shot is to a given query. MapNet helps establish the relation between the two different modalities (videos and query), which allows mapping of visual information to query space. After that, a deep reinforcement learning-based summarization network (SummNet) is developed to provide personalized summaries by integrating relatedness, representativeness and diversity rewards. These rewards jointly guide the agent to select the most representative and diversity video shots that are most related to the user query. Experimental results on a query-conditioned video summarization benchmark demonstrate the effectiveness of our proposed method, indicating the usefulness of the proposed mapping mechanism as well as the reinforcement learning approach.

关键词Query-conditioned Video Summarization Deep Reinforcement Learning Visual-text Embedding Temporal Modeling Vision Application
WOS记录号WOS:000460696500136
七大方向——子方向分类多模态智能
引用统计
被引频次:15[WOS]   [WOS记录]     [WOS相关记录]
文献类型期刊论文
条目标识符http://ir.ia.ac.cn/handle/173211/23598
专题复杂系统管理与控制国家重点实验室_先进机器人
通讯作者Yujia Zhang
作者单位1.Institute of Automation, Chinese Academy of Sciences
2.University of Chinese Academy of Sciences
3.Machine Learning Group, UiT The Arctic University of Norway
第一作者单位中国科学院自动化研究所
通讯作者单位中国科学院自动化研究所
推荐引用方式
GB/T 7714
Yujia Zhang,Michael Kampffmeyer,Xiaoguang Zhao,et al. Deep Reinforcement Learning for Query-Conditioned Video Summarization[J]. Applied Sciences - Basel,2019,9(4):750-765.
APA Yujia Zhang,Michael Kampffmeyer,Xiaoguang Zhao,&Min Tan.(2019).Deep Reinforcement Learning for Query-Conditioned Video Summarization.Applied Sciences - Basel,9(4),750-765.
MLA Yujia Zhang,et al."Deep Reinforcement Learning for Query-Conditioned Video Summarization".Applied Sciences - Basel 9.4(2019):750-765.
条目包含的文件 下载所有文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
applsci-09-00750.pdf(3674KB)期刊论文作者接受稿开放获取CC BY-NC-SA浏览 下载
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[Yujia Zhang]的文章
[Michael Kampffmeyer]的文章
[Xiaoguang Zhao]的文章
百度学术
百度学术中相似的文章
[Yujia Zhang]的文章
[Michael Kampffmeyer]的文章
[Xiaoguang Zhao]的文章
必应学术
必应学术中相似的文章
[Yujia Zhang]的文章
[Michael Kampffmeyer]的文章
[Xiaoguang Zhao]的文章
相关权益政策
暂无数据
收藏/分享
文件名: applsci-09-00750.pdf
格式: Adobe PDF
此文件暂不支持浏览
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。