CASIA OpenIR  > 毕业生  > 硕士学位论文
Thesis Advisor谭铁牛
Degree Grantor中国科学院研究生院
Place of Conferral北京
Degree Discipline计算机技术
Keyword短文本分类 深度学习 循环神经网络 长短时记忆单元
近年来随着深度学习相关研究的深入,相关方法在语音、图像及文本处理领域表现出很大优势,并分别在这几个领域的核心问题中取得了突破性进展。本文将结合深度学习算法的优势,从以下三个方面来解决传统算法在短文本分类问题上的不足。首先是改进单层神经网络的结构,对比现有的LSTM(Long-Short-Term-Memory)、GRU(Gated Recurrent Unit)等循环神经网络结构单元我们找到了适合短文本分类任务的结构单元,然后改进循环神经网络结构的输出。传统方法仅仅将最后一层的输出作为短文本的语义表示,本文采用卷积神经网络中的思想对循环神经网络的前向和后向输出进行融合,从而得到更好的短文本表示;其次,优化神经网络的输入和中间参数,结合词向量和自动编码机,分别对输入变量和网络结构做预训练,对比试验表明该预训练过程更有利于神经网络中的参数收敛,从而得到更好的分类效果;最后,本文引入一种改进的多层神经网络融合方法用于短文本分类,传统的深度神经网络只是简单地将单层神经网络的输出作为输入,一层一层叠加起来,本文借助LSTM中门限的思想,改进多层循环神经网络中层与层之间的联系,进一步优化短文本的语义表示,实验结果表明,改进后的多层神经网络分类效果要优于单层神经网络的分类效果。
Other Abstract

The rapid development of internet technology has enriched the ways of information access in recent years. With the popularity of microblog, twitter and other social media, many short texts including tweets, film reviews, instant messages and headlines are emerging every day. It can greatly facilitate information management and storage by building an auto-classification system of short texts. Furthermore, tasks like public opinion analysis and real-time hot spot analysis may be built upon in order to make it easier for people to get and understand information.

Traditional text classification systems are designed to manage long texts like books, documents and news. And the key algorithms of those systems are based on term frequencies and vector space models. Ideal outcomes cannot be seen simply by applying long text categorization algorithm to short texts, the main reason of which is that, on the one hand, short texts contain limited information, so the term frequency-based algorithm is not suitable for short text categorization; on the other hand, the existing algorithms tend to measure the match between the topic and key words, rather than base on the general understanding of the text.

In recent years, research on deep learning has advanced rapidly and related methods have shown great advantages in fields of speech recognition, image processing and natural language processing, and a breakthrough has been made in core problems of these areas. This paper will focus on problems of traditional algorithms in short texts classification from the following three parts, leveraging the advantages of deep learning.

First, we improve the structure of single layer neural network. By comparing the existing neural network units like Long-Short-Term-Memory (LSTM) and Gated-Recurrent-Unit (GRU) we found the most efficient unit for short text classification tasks. At the same time, we improve the outputs of Recurrent-Neural-Network (RNN) and their variations. Traditionally only the last output is used to classify texts, while adopting the pooling method, usually used in Convolutional-Neural-Networks (CNN), we merge all the outputs of a RNN in order to get a better representation of a short text. Next, we work out two ways to fine tune the inputs and middle parameters of our model. One is to optimize the input of the neural network with pre-trained word vectors. The other is to optimize the middle parameters in the neural network by using the auto-encoder. Our experiments show that these two methods can help the model converge to the global optimal point, thus greatly improve the results of short text classification tasks. Finally, we introduce an improved multi-layer recurrent neural network method for short text classifications. The traditional multi-layer neural network building is simply by taking the output of the former layer as the later layer’s input, layer upon layer. In this paper, drawing on the gate idea in LSTM, we improve the connections between different layers. By doing so, we get a more representative expression of short texts. Our experiments show that the performance of the improved multi-layer neural network is better than that of the single layer neural network.

Document Type学位论文
Recommended Citation
GB/T 7714
田俊. 基于深度学习的短文本分类研究[D]. 北京. 中国科学院研究生院,2016.
Files in This Item:
File Name/Size DocType Version Access License
基于深度学习的短文本分类研究-终稿.pd(2013KB)学位论文 暂不开放CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[田俊]'s Articles
Baidu academic
Similar articles in Baidu academic
[田俊]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[田俊]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.