Text2Video: An End-to-end Learning Framework for Expressing Text With Videos
Yang, Xiaoshan1,2; Zhang, Tianzhu1,2; Xu, Changsheng1,2
2018-09-01
发表期刊IEEE TRANSACTIONS ON MULTIMEDIA
卷号20期号:9页码:2360-2370
文章类型Article
摘要Video creation is a challenging and highly professional task that generally involves substantial manual efforts. To ease this burden, a better approach is to automatically produce new videos based on clips from the massive amount of existing videos according to arbitrary text. In this paper, we formulate video creation as a problem of retrieving a sequence of videos for a sentence stream. To achieve this goal, we propose a novel multimodal recurrent architecture for automatic video production. Compared with existing methods, the proposed model has three major advantages. First, it is the first completely integrated end-to-end deep learning system for real-world production to the best of our knowledge. We are among the first to address the problem of retrieving a sequence of videos for a sentence stream. Second, it can effectively exploit the correspondence between sentences and video clips through semantic consistency modeling. Third, it can model the visual coherence well by requiring that the produced videos should be organized coherently in terms of visual appearance. We have conducted extensive experiments on two applications, including video retrieval and video composition. The qualitative and quantitative results obtained on two public datasets used in the Large Scale Movie Description Challenge 2016 both demonstrate the effectiveness of the proposed model compared with other state-of-the-art algorithms.
关键词Multimedia Storytelling Video Analysis Deep Learning
WOS标题词Science & Technology ; Technology
DOI10.1109/TMM.2018.2807588
关键词[WOS]ANNOTATION ; REPRESENTATION ; NARRATIVES ; MOVIE ; WEB ; TV
收录类别SCI
语种英语
项目资助者National Natural Science Foundation of China(61432019 ; Beijing Natural Science Foundation(4172062) ; Key Research Program of Frontier Sciences, CAS(QYZDJ-SSW-JSC039) ; 61572498 ; 61532009 ; 61702511 ; 61720106006 ; 61711530243)
WOS研究方向Computer Science ; Telecommunications
WOS类目Computer Science, Information Systems ; Computer Science, Software Engineering ; Telecommunications
WOS记录号WOS:000442358200010
引用统计
文献类型期刊论文
条目标识符http://ir.ia.ac.cn/handle/173211/20467
专题模式识别国家重点实验室_多媒体计算与图形学
作者单位1.National Lab of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences
2.University of Chinese Academy of Sciences
推荐引用方式
GB/T 7714
Yang, Xiaoshan,Zhang, Tianzhu,Xu, Changsheng. Text2Video: An End-to-end Learning Framework for Expressing Text With Videos[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2018,20(9):2360-2370.
APA Yang, Xiaoshan,Zhang, Tianzhu,&Xu, Changsheng.(2018).Text2Video: An End-to-end Learning Framework for Expressing Text With Videos.IEEE TRANSACTIONS ON MULTIMEDIA,20(9),2360-2370.
MLA Yang, Xiaoshan,et al."Text2Video: An End-to-end Learning Framework for Expressing Text With Videos".IEEE TRANSACTIONS ON MULTIMEDIA 20.9(2018):2360-2370.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
Text2Video.pdf(2281KB)期刊论文作者接受稿开放获取CC BY-NC-SA浏览 请求全文
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[Yang, Xiaoshan]的文章
[Zhang, Tianzhu]的文章
[Xu, Changsheng]的文章
百度学术
百度学术中相似的文章
[Yang, Xiaoshan]的文章
[Zhang, Tianzhu]的文章
[Xu, Changsheng]的文章
必应学术
必应学术中相似的文章
[Yang, Xiaoshan]的文章
[Zhang, Tianzhu]的文章
[Xu, Changsheng]的文章
相关权益政策
暂无数据
收藏/分享
文件名: Text2Video.pdf
格式: Adobe PDF
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。