CASIA OpenIR  > 毕业生  > 硕士学位论文
Thesis Advisor曾毅
Degree Grantor中国科学院研究生院
Place of Conferral北京
Keyword小样本学习 脉冲神经网络 脑皮层 层次时序记忆模型
    传统人工神经网络,尤其是深度学习(Deep Learning,DL)所代表的人工神经网络(Artifitial Neural Networks,ANN)在进行模式识别任务时,强调通过定义损失函数,并通过数据的不断迭代训练,对神经元间的连接权重进行调整,从而完成学习任务。然而,人类并不是采用这种方式习得知识的,从生物学上来讲,传统人工神经网络所考虑的机理过于简单,并不能解释人类需要小样本的训练即可习得新知识的现象。本文着眼于具有生物学可解释性的神经元模型进行文本表征、并通过构建类脑皮层神经计算模型,进行文本生成、自
    首先,本文基于脉冲神经网络,将脉冲发放时间作为模式表示的特征,提出了基于齐次Poisson过程的神经编码方法、基于LIF 神经元模型的二值词表征转化方法、基于Izhikevich神经元模型的二值词表征转化方法等多种类脑无监督表示学习方法,将原始稠密的词向量转化为对大脑皮层友好的二值词表征。基于GloVe 及Word2Vec 两个不同源的词向量在五个数据集上进行试验,结果表明,所提出的二值词表征方法不仅在可解释性上胜于传统方法,还在语义相似性、文本分类等相关测试中超越传统方法。
Other Abstract
    Traditional neuronal networks, especially for deep learning techniques, can get good performance on several pattern recognition tasks under the condition that huge number of parameters and mass of data are available for training. They always define a cost function and fine-tune the weight via iterative training. However, human beings do not learn in this way. From the perspective of biology, the mechanism behind traditional neural networks may be too simple to explain that people can learning something new by means of few samples. This paper focus on neuron coding inspired binary word embeddings and brain-inspired neuronal computation models, and utilize them on one-shot learning tasks, such as text generation and question answering.
    We firstly propose homogeneous Poisson process based neural coding, spiking neural networks such as Leaky Integrate-and-Fire and Izhikevich neural model based binary coding post-processing algorithms, trying to convert dense original word embeddings into binary embeddings. We do experiments on two different word embeddings, GloVe and Word2Vec. The proposed methods are better than traditional methods in word similarity and word analogy tasks.
    We also propose a neocortex based computational model, Semantic Hierarchical Temporal Memory model (SHTM), for one-shot text generation. The model is refined from Hierarchical Temporal Memory model, which takes cell columns, distal dendrites, neuron models into consideration. We get binary embeddings via spiking neural network models proposed before and SHTM model for one-shot learning tasks. LSTM is used for comparative study. Results on three public datasets show that our model performs much better than LSTM on the measures of mean precision and BLEU score. In addition, we utilize our model to do question answering in the fashion of text generation and verify its
Subject Area模式识别与智能系统
Document Type学位论文
Recommended Citation
GB/T 7714
王寓巍. 类脑皮层表征神经计算模型及其应用[D]. 北京. 中国科学院研究生院,2017.
Files in This Item:
File Name/Size DocType Version Access License
YuweiWangThesis0601F(8671KB)学位论文 暂不开放CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[王寓巍]'s Articles
Baidu academic
Similar articles in Baidu academic
[王寓巍]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[王寓巍]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.