CASIA OpenIR  > 毕业生  > 博士学位论文
面向网络空间的用户行为建模研究
余峰
Subtype博士
Thesis Advisor谭铁牛
2020-05-31
Degree Grantor中国科学院自动化研究所
Place of Conferral中国科学院自动化研究所
Degree Discipline模式识别与智能系统
Keyword用户行为建模 高阶特征交互 时序建模 目标注意力 关联记忆
Abstract

面向网络空间的用户行为建模是指通过表示学习的方式,将网络空间中的用户行为编码为计算机可以处理的向量形式。这是实现用户行为分析和理解的最基础同时也是最重要的步骤之一。高质量的用户行为建模可以使计算机有效地完成网络空间中与用户相关的各种任务,如计算广告、电子商务、社交媒体分析等,因此本论文在理论和应用上都具有重要的意义和价值。

对于用户行为建模,行为数据中模式和信息的有效挖掘极为重要。本论文围绕如何有效地挖掘用户行为数据中的模式和信息来展开,重点关注行为中三种信息的建模:属性建模、序列建模和结构信息建模。

论文的主要工作和创新点归纳如下:

1. 提出了基于高阶特征生成和表示学习的行为属性建模方法

用户行为可以由时间、地点、人口特征等多个属性变量描述,为了更好地学习和理解用户的行为,建模多个属性变量之间的交互是一个有效的途径。传统的方法先将属性变量转换为稠密的特征向量,再通过特征向量之间的内积大小表示特征交互的重要程度。但是由于特征交互的复杂度随着特征阶数的增加呈指数级增长,所以已有的特征交互算法一般只能用于学习特征间的低阶交互,无法高效地学习特征间的高阶交互。为此,我们引入牛顿恒等式,将特征交互所需要的向量多项式乘积的形式简化成向量的幂和,把指数复杂度降低到线性复杂度,进而可以高效地学习特征间的任意高阶交互。

另一方面,表征向量可以赋予特征更多的泛化能力和表达能力。对于特征交互得到的高阶特征,如果能够学习出有效的表征向量,将会给模型带来更多的泛化能力和表达能力。为了学习高阶特征的表征向量,我们提出了张量特征交互网络,通过张量和两个低阶特征向量的乘法,学习出对应的高阶特征向量。

2. 提出了基于循环神经网络和注意力卷积网络的行为序列建模方法

网络空间中用户连续的多个行为存在很强的时序关联,如何利⽤行为间的转移模式来更好地建模用户行为是⼀个重要的挑战。用户行为间的转移可以反映用户行为背后的意图或者兴趣,用户的兴趣往往是随时间动态变化的。现有的方法只是建模用户的一般兴趣或者连续两次行为之间的关系,这样就很难同时满足行为建模实时性和准确性的要求。因此,我们提出了动态循环神经网络,根据用户当前的行为不断更新用户的动态兴趣,对用户当前的动态兴趣进行建模可以极大地提高模型实时推荐的准确性。

另一方面,我们还提出方法来应对行为序列中行为数量大且噪声多的两大挑战。对于数据量大的问题,我们使用行为表示学习模块来学习整体行为的表征向量。对于大量噪音的问题,我们使用内容注意力和时间注意力,学习每次行为的内容和时间重要性权重,关注重要的行为。因此,我们提出了融合注意力机制的卷积神经网络模型。其中,行为表示学习模块和注意力机制可以学习出好的行为表征向量,卷积神经网络可以灵活地提取出输入中的重要特征,然后建模特征之间的高阶交互。

3. 提出了融合关联记忆和目标注意力的行为结构信息建模方法

为了更加准确地建模用户行为,我们还引入了全局关联的结构信息。传统的方法只能建模用户的行为序列,很难获取用户细粒度的偏好。考虑到物品与类别和风格等属性之间存在着非常丰富的联系,我们提出了融合知识的关联记忆网络模型,通过引入物品属性的知识图谱,可以很好地补充物品之间的联系,利用结构化的实体信息来增强我们对用户行为的理解,从而更好地实现用户行为建模。

另一方面,考虑到不同的用户行为序列可以通过相同的行为连接成行为图,我们可以将不同序列中的行为转移关系合并到一张图上,得到全局的行为转移关系图,反映全局的行为和兴趣转移。另外,考虑到用户兴趣和候选物品的多样性,我们提出了融合目标注意力的图神经网络模型,目标注意力可以自适应地激活用户兴趣中与目标物品相关的部分,进而实现对用户行为更加全面而精准地建模。

Other Abstract

User behavior modeling in cyberspace is to encode user behaviors into vectors via representation learning, which can be easily processed by computer. This is one of the most fundamental and important steps in user behavior analyses and understanding. High-quality user behavior modeling enables computers to effectively perform user-related tasks in cyberspace, such as computational advertising, e-commerce, social media analyses, etc. Therefore, this research is of great significance and value in both theory and application.

For user behavior modeling, it is very important to discover the patterns and information within, between and beyond behaviors. This paper aims to effectively mine patterns and information of user behaviors, focusing on three types of behavior information: attribute features, behavior sequences, and structure information in behaviors.

The main contributions are summarized as follows:

1. Modeling attribute features based on high-order feature generation and representation learning

User behavior can be described by attribute features, such as time, place, population and so on. Modeling the feature interactions help understand users' behaviors. Traditional methods convert these categorical features to dense features vectors, then count the importance of feature interactions by the inner product of feature vectors. Because the complexity of feature interactions increases exponentially with the order of features, existing methods only learn low-order feature interactions. Therefore, we introduce Newton's identities to simplify the polynomials into the power sums, reducing the exponential complexity to linear complexity, so that we can learn any higher-order feature interactions efficiently.

In addition, feature representation has a better capacity of generalization and expression. User behaviors modeling will be more generalized and expressive if we can learn the representation vectors of high-order features. In order to learn the representation vectors of high-order features, we propose a tensor-based feature interaction network, which transforms every two low-order feature vectors into corresponding higher-order feature vectors using tensor operation.

2. Modeling sequential behaviors based on recurrent neural network and attention-based convolutional neural network

There are strong temporal correlations among users' sequential behaviors, so it is challenging to make use of the transition patterns among behaviors to better learn user behaviors. The transitions among user behaviors reflects the intentions or interests of users, which often change with time. The existing methods only model general interests of users or the relationship between two consecutive behaviors, making it difficult to meet the demand of real-time accuracy. Therefore, we propose a dynamic recurrent neural network, which can update the dynamic interests of users according to both the current user behavior and historical states. The modeling of dynamic interests of users can greatly improve the accuracy of real-time behavior-modeling systems.

Furthermore, we propose a model to alleviate problems of large amount of data and noise data in user behaviors. For the problem of large amount of data, we utilize a behavior representation learning module to learn the representation vector of overall behaviors. For the problem of noise data, we use content attention and temporal attention, learning the importance weight of content and time for each behavior. We propose an attention-based convolutional neural network model, which can learn content attention and temporal attention for each behavior, selecting key behaviors from large number of noisy behaviors. Behavior representation learning module and attention mechanism can learn good representation vectors of behaviors. The convolutional neural network can flexibly extract important features from the input and then model high-level features.

3. Modeling structure information of behaviors based on target attentive graph neural network and associative memory network

In order to model user behaviors more accurately, we also introduce globally associative structure information. It is difficult to obtain users' fine-grained preferences by only modeling user's behavior sequences. Considering that there are very rich relations among items and attributes, we propose associative memory network model that integrates knowledge graph. The attributes of items from the knowledge graph can be a good complement of the limited links between items in sequences. And the structured entity information is used to enrich the understanding and realize better modeling of user behaviors.

Observing that the user's behavior sequences in different time periods can complement each other, we can integrate the sequences in different time periods into one graph, which reflects global transitions among behaviors and interests. In addition, considering the diversity of user interests and candidate items, we propose a target attentive graph neural network model. Target attention can adaptively activate the parts of interests related to target items, realizing comprehensive and accurate modeling of user behaviors.

Pages128
Language中文
Document Type学位论文
Identifierhttp://ir.ia.ac.cn/handle/173211/39034
Collection毕业生_博士学位论文
Recommended Citation
GB/T 7714
余峰. 面向网络空间的用户行为建模研究[D]. 中国科学院自动化研究所. 中国科学院自动化研究所,2020.
Files in This Item:
File Name/Size DocType Version Access License
Thesis无电签.pdf(6123KB)学位论文 暂不开放CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[余峰]'s Articles
Baidu academic
Similar articles in Baidu academic
[余峰]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[余峰]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.