毕业生知识库

    传统人工神经网络，尤其是深度学习（Deep Learning，DL）所代表的人工神经网络（Artifitial Neural Networks，ANN）在进行模式识别任务时，强调通过定义损失函数，并通过数据的不断迭代训练，对神经元间的连接权重进行调整，从而完成学习任务。然而，人类并不是采用这种方式习得知识的，从生物学上来讲，传统人工神经网络所考虑的机理过于简单，并不能解释人类需要小样本的训练即可习得新知识的现象。本文着眼于具有生物学可解释性的神经元模型进行文本表征、并通过构建类脑皮层神经计算模型，进行文本生成、自

动问答等小样本学习任务。

    首先，本文基于脉冲神经网络，将脉冲发放时间作为模式表示的特征，提出了基于齐次Poisson过程的神经编码方法、基于LIF 神经元模型的二值词表征转化方法、基于Izhikevich神经元模型的二值词表征转化方法等多种类脑无监督表示学习方法，将原始稠密的词向量转化为对大脑皮层友好的二值词表征。基于GloVe 及Word2Vec 两个不同源的词向量在五个数据集上进行试验，结果表明，所提出的二值词表征方法不仅在可解释性上胜于传统方法，还在语义相似性、文本分类等相关测试中超越传统方法。

    其次，本文提出基于语义化层次时序记忆模型的小样本学习模型。该工作结合受大脑皮层相关中神经元、远端突触、细胞柱及时序计算机制的启发的层次时序记忆模型，建立神经计算模型，在完善原始模式表征方式的同时，结合脉冲神经网络所转化的二值词表征，进行小样本学习任务，并在文本生成、自动问答等任务的三个数据集上取得了较传统模型LSTM模型更好的效果。

    Traditional neuronal networks, especially for deep learning techniques, can get good performance on several pattern recognition tasks under the condition that huge number of parameters and mass of data are available for training. They always define a cost function and fine-tune the weight via iterative training. However, human beings do not learn in this way. From the perspective of biology, the mechanism behind traditional neural networks may be too simple to explain that people can learning something new by means of few samples. This paper focus on neuron coding inspired binary word embeddings and brain-inspired neuronal computation models, and utilize them on one-shot learning tasks, such as text generation and question answering.
    We firstly propose homogeneous Poisson process based neural coding, spiking neural networks such as Leaky Integrate-and-Fire and Izhikevich neural model based binary coding post-processing algorithms, trying to convert dense original word embeddings into binary embeddings. We do experiments on two different word embeddings, GloVe and Word2Vec. The proposed methods are better than traditional methods in word similarity and word analogy tasks.
    We also propose a neocortex based computational model, Semantic Hierarchical Temporal Memory model (SHTM), for one-shot text generation. The model is refined from Hierarchical Temporal Memory model, which takes cell columns, distal dendrites, neuron models into consideration. We get binary embeddings via spiking neural network models proposed before and SHTM model for one-shot learning tasks. LSTM is used for comparative study. Results on three public datasets show that our model performs much better than LSTM on the measures of mean precision and BLEU score. In addition, we utilize our model to do question answering in the fashion of text generation and verify its

superiority.