CASIA OpenIR  > 毕业生  > 博士学位论文
多模态跨平台社会事件分析技术研究
钱胜胜
学位类型工学博士
导师徐常胜
2017-05-31
学位授予单位中国科学院研究生院
学位授予地点北京
关键词多媒体 社会事件分析 多模态 跨平台 大规模
摘要

社会事件是指发生在特定地点和特定时间的特定行为,它由许多随时间推移的子事件组成。随着互联网的快速发展,出现越来越多的社会媒体网站,用户可以非常方便地在这些网站上分享其想法、图片、帖子和其他相关活动。因此,当一个流行事件发生在我们周围时,它可以在不同社会媒体网站中快速地进行传播,同时会产生大量的多媒体数据。用户上传的大部分与事件相关的多媒体内容都与某些特定的话题相关,如果对这些数据进行人工识别和聚类来获得真实场景中事件的整个主题演变过程,将是非常耗时的。因此,一个社会事件分析的通用框架是非常重要和必要的,它能够及时地了解社会事件随时间演变的发展趋势。
然而,由于社会事件数据来自不同的网站,具有跨平台、多模态、大规模和噪声大等属性,进行社会事件分析的研究非常具有挑战性。本文的研究目标是设计一个通用的社会事件分析框架来解决上述几个问题,并针对社会事件表示、检测、跟踪和演变分析任务,构建一个有效鲁棒的社会事件分析框架。在提出的框架中,我们针对每个任务设计了具体的模型,详细来说,本文的主要贡献体现在如下5个方面:

1. 社会事件表示的目的是从大量媒体数据中抽取有效的特征,得到语义层面的多模态事件表示。针对社会多媒体数据多模态和跨平台特性,我们提出一种基于非参数贝叶斯字典学习模型的多模态跨平台协同学习事件表示方法。该方法能够充分利用多模态跨平台的数据源,在事件表示过程中进行互相补充和互相促进。
2. 社会事件检测的目的是从大规模的社会媒体数据自动地挖掘和识别有意义的社会事件。针对大规模社会媒体数据的监督属性,我们提出了一种新颖的提升多模态有监督潜在狄利克雷分布模型,该模型在boosting框架中引入有监督主题模型,能够适合大规模数据的分析。而且,提出的方法能够联合利用多媒体数据的多模态信息和监督信息,用于社会事件分析。
3. 社会事件跟踪的目的是在时序性社会事件数据中跟踪多个社会事件。如何建模时序性多模态数据以及避免跟踪过程中模型漂移是事件跟踪过程中的两个难点。我们提出了一种新颖的在线多模态多专家学习方法来建模时序性多模态社会事件数据。而且,提出的方法采用了一种新颖的多专家最小化恢复方案,并允许模型剔除不满意的模型并更新当前最有效的模型,这样可以有效地解决跟踪模型漂移问题,提高跟踪精度。
4. 社会事件演变分析的目的是从时序性多模态事件数据集来获得整个社会事件的演变过程,以及得到随时间推移的事件摘要细节。为了实现这个目标,我们提出了一种新颖的多模态事件主题模型,该模型可以高效地建模多模态媒体数据,并且可以区分视觉代表性主题和非视觉代表性主题。为了将该模型应用于社会事件演变分析中,本文采用一种新颖的增量学习策略,其可以随时间获得社会事件的时序性文本和视觉主题,从而帮助理解社会事件。
5. 细粒度社会事件分析的目的是从多个社会媒体源进行社会事件的细粒度主题和观点挖掘。为了实现这个目标,我们提出了一种新颖的多模态多视角主题观点挖掘方法,该方法能够同时有效地考虑多模态属性和多视角属性来进行多个数据源的主题和观点学习。而且,该方法能够从所有数据源中发现多模态数据的共同主题,总结出每个特定主题的相似和不同之处,而且能在不同数据源学习的主题中自动挖掘多视角观点。

其他摘要Social event is something that occurs at a specific place and time associated
with some specific actions, and it consists of many stories over time. With the
rapid development of Internet, more and more social networking sites appear
and allow users to conveniently share their ideas, pictures, posts, and activities.
Therefore, when a popular event is happening around us, it can spread very
fast in different social media sites with substantial amounts of multimedia data
including images, videos, and texts. Most of these multimedia contents associated
with social events uploaded by users are related to some specific topics, and it
is very time-consuming for people to manually identify or cluster them to obtain
the whole topic evolution of an event in real world scenarios. Therefore, it is
important and necessary to propose the framework of social event analysis to
know the evolutionary trend of social event over time automatically.
However, it is very challenging to do social event analysis because social
event data from different social media sites have multi-platform, multi-modal,
large-scale, and noisiness characteristics. The goal of this thesis is to design
advanced multimedia techniques to deal with the above issues and establish an
effective and robust social event analysis framework for social event representation,
detection, tracking and evolutionary analysis. In the proposed framework,
we have designed the specific model for each task. Specifically, we have made the
following contributions:

1. Social event representation. It aims to learn the semantic-based multimodal
event description by exploiting the multi-modal data from multiplatform
social media resources, including user-provided textual information
and visual information. To achieve this goal, we propose a
generic cross-platform collaborative learning algorithm based on the nonparametric
Bayesian dictionary learning model. As a result, our model
can effectively make use of the virtues of different information sources to complement and enhance each other for feature representation.
2. Social event detection. It aims to automatically mine and identify social
events from large-scale social media data. To exploit the supervised category
information of the large-scale data, we propose a novel boosted multimodal
supervised Latent Dirichlet Allocation (BMM-SLDA) by integrating
a supervised topic model in the boosting framework, which can effectively
process large-scale data iteratively. Furthermore, the proposed model can
effectively exploit the multi-modality and the multi-class property of social
events jointly.
3. Social event tracking. It aims to track the topic evolution of multiple events
over time in the social media resource. Due to the multi-modality of timeseries
data and the model drift of the event tracking in the social event analysis,
we propose a novel online multi-modal multi-expert learning method
(OMMEL) to model online multi-modal social event data. The proposed
model adopts a novel multi-expert minimization restoration scheme and
allows the tracked model to evolve backwards to undo undesirable model
updates, which can help alleviate the model drift problem of social event
tracking.
4. Social event evolution analysis. It aims to utilize time series multi-modal
event data to obtain the evolutionary trends of social events and generate
effective event summary details over time. To achieve this goal, we propose
a novel topic model method (multi-modal Event Topic Model, mmETM) to
effectively fuse multi-modal information and consider visual-representative
topics and non-visual-representative topics together. Based on the mmETM,
a novel incremental model is proposed for social event evolution analysis,
which can obtain the whole topic evolutionary process of events with
textual and visual topics over time and help understand the events.
5. Social event fined-grained analysis. It aims to conduct the fined-grained
topic and opinion analysis of the social event from multiple social media
resources. To achieve this goal, we propose a novel multi-modal multiview
topic-opinion mining (MMTOM) model by considering multi-view and
multi-modal properties for topic and opinion learning in multiple collection
sources. Furthermore, the proposed model is able to not only discover
multi-modal common topics from all collections as well as summarize the
similarities and differences of these collections along each specific topic, but
also automatically mine multi-view opinions on the learned topics across
different collections.


学科领域模式识别与智能系统
文献类型学位论文
条目标识符http://ir.ia.ac.cn/handle/173211/14781
专题毕业生_博士学位论文
作者单位Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China
推荐引用方式
GB/T 7714
钱胜胜. 多模态跨平台社会事件分析技术研究[D]. 北京. 中国科学院研究生院,2017.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
多模态跨平台社会事件分析技术研究.pdf(38904KB)学位论文 暂不开放CC BY-NC-SA请求全文
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[钱胜胜]的文章
百度学术
百度学术中相似的文章
[钱胜胜]的文章
必应学术
必应学术中相似的文章
[钱胜胜]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。