中国科学院自动化研究所机构知识库

互联网的高速腾飞让大众进入网络世界，社会媒体在互联网的沃土上蓬勃发展。社会媒体应用已经渗透到人们生活的各个角落，影响着人们的日常生活。用户是社会媒体内容的生产者，也是其服务的终极目标。互联网上的各种应用为用户提供便捷服务的同时，也记录着用户的行为数据。用户行为是用户兴趣偏好的潜在表示，对用户行为进行建模，可以捕获用户在社会媒体中的兴趣偏好，支撑不同的下游应用，如个性化推荐系统、个性化搜索等。用户行为具有时序性，对用户在社会媒体中的序列行为建模，是有效跟进用户动态兴趣演变，捕捉用户长短期兴趣偏好的重要手段。此外，目前用户序列行为建模还存在几个关键性问题，如用户在社会媒体上产生的内容具有多模态性、用户行为的异构多义性、复杂的交互关系以及用户行为建模的典型应用缺乏可解释性。在本研究中，我们将从社会媒体用户序列行为建模的四个关键问题切入，对序列行为建模进行研究，提高用户兴趣建模的高效性和准确率，为下游应用提供有力的支撑。本文的研究内容和主要贡献如下：
1. 用户行为多模态内容建模研究。序列行为建模的基础是对用户行为中的多模态内容建模。社会媒体上充斥着多模态内容，比如文本、图像、视频、音频等。在设计服务于用户的各类应用时，多模态的内容是原材料，对多模态内容进行分析，提取和建模对任务有用的信息和模式，有利于挖掘底层信息以改进应用的表现。因此，本文研究了多模态内容建模的两个方面，为后续研究工作奠定基础。一是基于多模态内容对用户行为进行建模，根据用户在 Flickr 上发布的图片，设计显性和隐性特征，通过模型融合自动预测图片的流行度。二是基于多模态内容对用户关系进行建模，根据用户构建的 Twitter 列表中的元数据和关系数据，设计一组多模态相似度特征以构建有效的社会关系标注模型，辅助社交网络的构建。
2. 上下文感知的异构多义序列行为建模。用户的序列行为可以揭示用户兴趣演变，序列推荐是面向用户的在线服务的一项重要任务。社会媒体上用户行为存在许多特性：行为异构性、行为多义性、行为上下文依赖性。本文提出了一种用户序列行为建模算法，该算法是一个上下文感知的自注意力网络。模型中考虑了异构用户行为，并将其投影到一个公共的潜在语义空间中，然后将隐层输出输入到特征级自注意力网络中，捕捉用户行为的多义性。模型采用前向和后向位置编码矩阵来建模动态上下文依赖关系。同时，自注意力网络可以对序列建模进行并行运算，有效地捕捉用户行为的动态兴趣表示的同时，能极大提高模型的运算效率。
3. 交互异构网络辅助的综合序列行为建模。现有的序列推荐算法通常使用历史行为的简单序列关系来建模用户偏好，其仅考虑关于用户行为的局部信息，以至于它在捕获复杂的用户偏好方面具有有限的表示能力。本文提出了一种基于元路径的上下文感知序列推荐方法，将异构信息网络和元路径机制引入推荐系统中，构建了用户与项目之间隐藏依赖关系的全局上下文信息，有助于在序列推荐系统中合理利用网络中的密集连接来拓展用户的兴趣。算法采用自注意力网络建模用户局部兴趣偏好，采用异构信息网络和元路径建模用户全局兴趣偏好，采用协同注意力模型捕捉用户历史行为和上下文信息之间复杂的交互，从而增强了对用户偏好的综合建模。
4. 知识图谱推理的可解释性序列行为建模。当前基于神经网络的方法在推荐任务上都取得了很高的准确率，然而大多数方法都没有考虑在做出推荐决策的同时为用户提供一个合理的解释。本文提出了一种交互驱动的可解释性用户建模和推荐算法，通过多模态融合引入知识图谱中的结构知识，为用户和项目提供更好的表达，通过联合学习提高推荐系统的性能。模型从知识图谱中提取用户和项目之间的语义路径，并通过交互表示模块对相应语义路径进行编码，来学习用户-项目交互的语义表示。序列交互建模模块对用户-项目交互序列进行编码，目的在于捕捉用户的动态偏好漂移。算法在提升推荐系统准确率的同时，也能为推荐结果给出合理的解释，赋予了推荐系统可解释能力。

The rapid development of the Internet allows the masses to enter the online world. Social media is flourishing on the fertile ground of the Internet. Social media applications have penetrated into people's lives, affecting people's daily lives. Users are the producers of social media content and the ultimate goal of their services. Various applications on the Internet provide users with convenient services while also recording user behavior data. User behavior is a potential expression of user interest preferences. Modeling user behaviors can capture user's interest preferences in social media and support different downstream applications, such as personalized recommendation systems and personalized search. User behaviors on social media change dynamically over time. Modeling on users' sequential behaviors in social media is an important means to effectively follow the evolution of users' dynamic interests and capture their long-term and short-term interest preferences. In addition, there are still several key issues in the modeling of user sequential behaviors, such as the multi-modality of the content generated by users on social media, the heterogeneity and polysemy property of user behaviors, the complex interaction between users and items, and the typical applications of models lack Interpretability. In this thesis, we aim to tackle four key issues of user sequential behavior modeling on social media, to improve the efficiency and accuracy of user interest capturing, and provide strong support for downstream applications. The major contributions of this thesis are as follows:

1. Research on multi-modal content modeling on social media. The basis of sequential behavior modeling is to model the multi-modal content of user behavior. Social media is full of multi-modal content, such as text, images, video, audio, and more. When designing various applications that serve users, multi-modal content is raw materials. Analysis of multi-modal content and the extraction of useful information for the task are beneficial to mining the underlying patterns to improve the performance of the applications. Therefore, we study two aspects of multi-modal content modeling, laying the foundation for follow-up research. The first one is to model user behaviors based on multi-modal content. We propose a method to automatically predict the popularity of pictures posted by users on Flickr by the stacking model and the specially designed explicit and implicit features. The second one is to model social relationships based on multi-modal content. We propose an effective social relationship labeling model with the specially designed multi-modal similarity features constructed from metadata and the relationship of users' Twitter list, to assist in the construction of social networks.

2. Contextual sequential behavior modeling based on user's heterogeneous and polysemy behaviors. A user's sequential behaviors can reveal the evolution of user interests. The sequential recommendation is an important task for user-oriented online services. There are many characteristics of user behaviors on social networks: user behaviors are inherently heterogeneous, user behaviors have the polysemy property, and user behaviors are dynamically context-dependent. we propose a unified contextual self-attention network to address the three properties. Heterogeneous user behaviors considered in our model are projected into a common latent semantic space. Then the output is fed into the feature-wise self-attention network to capture the polysemy of user behaviors. Besides, the forward and backward position encoding matrices are proposed to model dynamic contextual dependency. The proposed method can effectively capture the dynamic interest expression of user behavior, and can greatly improve the computing efficiency of the model.

3. Interactive heterogeneous information network enhanced comprehensive sequential behavior modeling. The existing sequential recommendation algorithms usually use historical sequential behaviors to model user preferences, which only take into account local information about user behaviors, so that it has limited expressive power in capturing complex user preferences. We propose a meta-path augmented context-aware sequence recommendation method, which introduces heterogeneous information networks and meta-path mechanism into the recommendation system, and builds global context that contains the potential dependencies between users and items, which is helpful for users to expand their interest through the dense connections in the network in a reasonable manner. The proposed approach introduces meta-paths from a heterogeneous information network to capture the global context information, and the position-based self-attention mechanism is adopted to model the local preference representation efficiently. Compared with the methods which only consider the local preference, our proposed method takes the advantages of incorporating global context information which extracts structural features that captures relevant semantics to construct users’ global preference representation for the sequential recommendation. We further adopt the co-attention mechanism to model complex interactions between the global context and users’ historical behaviors for better user representations. 

4. Explainable sequential behavior modeling based on knowledge graph (KG) reasoning. The current neural network-based methods have achieved high accuracy in recommendation tasks. However, most methods have not considered providing a reasonable explanation for users while making recommendation decisions. We propose a novel explainable interaction-driven user modeling algorithm to exploit the knowledge graph for constructing an effective and explainable sequential recommender. Qualified semantic paths between specific user-item pair are extracted from KG. Encoding those semantic paths and learning the importance scores for each path provides the path-wise explanation for the recommendation system. Different from traditional item-level sequential modeling methods, we capture the interaction-level user dynamic preferences by modeling the sequential interactions. It is a high-level representation that contains auxiliary semantic information from KG. Furthermore, we adopt a joint learning manner for better representation learning by employing multi-modal fusion, which benefits from the structural constraints in KG and involves three kinds of modalities. The proposed algorithm achieves impressive performance in terms of endowing sequential recommendation systems with accuracy and Explainability.