基于深度语义特征表示的短文本情感分析研究

CASIA OpenIR > 毕业生 > 硕士学位论文

	基于深度语义特征表示的短文本情感分析研究
	施伟
	2016-05
学位类型	工程硕士
中文摘要	信息时代的来临，信息技术的不断更新换代，互联网的普及，使得社交网络、电子商务急速发展。互联网络已经成为人民生活中获取、交流信息和情感交流的重要方式。以豆瓣、微博、twitter等形式为主的带有主观情感倾向的信息迅速膨胀。由于这些信息的长度有限、表达简练、情感集聚、我们称之为短文本。由于短文本的简洁性和不规范性，利用传统方法过多依赖于人工特征设计以及文本的预处理工具，结果受到一定的限制。而且传统的基于人工构建特征的方法没有考虑文本的语序信息，不利于对文本中情感信息进行抽取。为解决以上问题，本文在传统循环神经网络 (Recurrent Neural Network, RNN)网络模型的基础上，引入长短期记忆单元(Long Short Term Memory, LSTM)，从增加序列未来信息、增强语义学习和添加注意力机制等三个方面对其进行改进，提出了双向多层LSTM网络分类模型、LSTM自编码网络分类模型和带注意力机制的LSTM自编码联合学习模型。主要研究内容包括： 1）提出了基于词向量的多层双向LSTM网络框架。传统基于词袋的支持向量机(Support Vector Machine, SVM)方法和卷积神经网络 (Convolutional Neural Network, CNN)分类方法没有考虑时间序列、历史信息。LSTM-RNN将短文本以词为单位转化为词向量输入网络，学习短文本的深层语义表示，再将其进行分类。双向LSTM在一定程度上解决了时间序列首尾权重不平衡的问题，同时融合了历史和未来信息，使得生成的句子表示包含语义更为完全，但仍然无法完整包含整句的语义和标签信息。实验结果表明双向LSTM网络能够获取更多的信息，生成的语义表示具有更好的辨识性，分类效果较SVM，CNN以及RNN有较大的提升。 2）为了加强分类模型中语义的学习过程，得到句子的更为完整的深层语义表示。本文提出了LSTM和自编码（Auto Encoder）的联合学习模型，有监督训练的同时，加入自编码网络(Auto Encoder)对句子进行编解码，生成包含语义和标签信息的句子表示。实验结果表明，自编码网络的加入使得分类效果在中英文数据集上均有所提高，联合训练效果也较离线训练要好。 3）人脑在接受文本或者图像信息的时候，在了解整体信息之后会将注意力集中在某些词和图像的片段上进行加深印象，情感分类也是如此，在掌握整句语义和标签信息后，需要对某些关键词，例如转折词、情感词等，进行聚焦来把握整句的情感倾向。因此本文在联合学习模型的基础上提出了三种模式的注意力机制(Attention Mechanism)，旨在能够将网络自动聚焦在某些关键的词或者字句的语义表示上面，生成信息更加丰富并且关键词突出的深度语义特征表示，实验中三种注意力机制的分类效果都较LSTM +AutoEncoder联合学习要好，其中基于隐层输出的注意力机制效果最好。
英文摘要	The coming of information era, updating of information technology, popularization of mobile Internet, help social networks and e-commerce boom. Internet has been the most important way for people to obtain, exchange information and communicate. Information with personal sentiments in social networks such as BBS, Microblog, and Twitter, expand rapidly. Due to their limited length, concise expression and Emotional concentration, they are called Short Texts. On account of short texts’ conciseness and irregularities, methods that rely too much on handcrafted features and pre-processed tools, have been restricted. And traditional methods based on handcrafted features, have not taken words’ order into consideration, which is bad for extracting texts’ sentiment information. Aiming at these drawbacks, this thesis introduced Long Short Term Memory (LSTM) cell into traditional RNN network, improved the model in three aspects: adding sequence information, enhancing semantic learning and attention mechanism, and proposed Bidirectional Multi-layer LSTM Network, LSTM AutoEncoder Network, and Attention-based LSTM AutoEncoder joint learning model. Main research contents include: Proposed a multi-level bidirectional LSTM network. Traditional Support Vector Machine (SVM) method based on Bag of Words and Convolutional Neural Network (CNN) have not taken temporal order and history information into account. The proposed model in this thesis combines words’ vector and gets sentences’ deep semantic representations. Bidirectional LSTM Neural Network has alleviated the problem of imbalanced weights for the beginning and ending words in a sentence, and has made the semantic representation of the sentence more complete by merging the history and future information. Experiments show that bidirectional LSTM network outperforms SVM, CNN and LSTM a lot in fine-grained classification. To enhance the learning of semantic information and get sentence’s integrated deep semantic representation. This thesis proposed a joint learning model of LSTM and Auto Encoder. In the process of supervised learning, the AutoEncoder could encoder the sentence without supervision and get the more comprehensive representations which include both the semantic and label information. We got a better performance compared to the bidirectional LSTM network on both the English and Chinese datasets. Results show that joint learning is also better than off-line in my experiments. As far as we know, when our human brain receives the information from texts and images, we would focus on some specific words or parts of the images after getting the overall features, it is called “Attention Mechanism”. Similarly, sentiment analysis also needs to focus on the key words such as emotional words and adversatives after getting the information of the entire sentence. So in this thesis, I proposed 3 kinds of attention mechanisms to focus on key words and sub-sentence on the foundation of the joint learning model of LSTM and Auto Encoder to get more abundant and highlighted semantic representation. In the experiments, all the 3 kinds of Attention model outperform the LSTM-AutoEncoder networks, and the one based on hidden units’ outputs gets the best result in the fine-grained classification.
关键词	情感分类 Lstm 自编码注意力机制
文献类型	学位论文
条目标识符	http://ir.ia.ac.cn/handle/173211/11703
专题	毕业生_硕士学位论文
作者单位	中国科学院自动化研究所
第一作者单位	中国科学院自动化研究所
推荐引用方式 GB/T 7714	施伟. 基于深度语义特征表示的短文本情感分析研究[D]. 北京. 中国科学院大学,2016.

条目包含的文件
文件名称/大小	文献类型	版本类型	开放类型	使用许可
基于深度语义特征表示的短文本情感分析研究（2958KB）	学位论文		限制开放	CC BY-NC-SA