毕业生知识库

    传统的人工神经网络(Artificial Neural Network, ANN)目前在计算机视觉和自然语言处理等模式识别任务中表现出卓越的性能，但是鉴于其依赖于大量的训练样本和强大的算力基础并且增量学习和自适应能力比较弱，以深度神经网络(Deep Neural Network, DNN)为基础的一些计算模型依然有许多亟待改进的地方。近年来脉冲神经网络(Spiking Neural Network, SNN)以其更加类生物的结构和计算方式受到了许多研究者的关注，并且因为其可以更多地与哺乳动物大脑的层次化结构和多种可塑性规则相结合，SNN被认为更有潜力突破当前人工智能方法鲁棒性不足和计算开销巨大的问题。与此同时，大脑中好奇这类的机制和所涉及的认知通路在我们主动、高效的终生学习过程中起到至关重要的作用。因此本文将以SNN的优化为基础，结合好奇机制，探索其与SNN结合的方式及其认知环路的计算建模，主要成果如下：

    1. 提出了多种生物可塑性机制启发的脉冲神经网络优化方法

    从SNN学习算法角度而言，不同于将SNN的激活函数变换为类似ANN中连续可微的形式，或者将原始脉冲序列转换成放电率运算，本文直接建模大脑神经元的稳态膜电位调控机理，提出了以电压为中心的优化算法(Voltage-driven Plasticity-centric SNN, VPSNN)，并在MNIST数据集上进行了验证，得到98.52%的分类准确率，为当下纯生物可塑性优化SNN的最好结果。

    2. 构建了融合好奇机制的脉冲神经网络模型

    从SNN效率提升角度而言，在传统调优的基础上，本文受大脑中好奇机制的特点和优势启发，提出了一种有效地和目前SNN学习规则融合的模型Curiosity-based Spiking Neural Network (CBSNN)，并在MNIST、Iris、NET talk、Fashion-MNIST和CIFAR-10数据集上进行了验证，达到同等分类正确率的前提下，CBSNN只用VPSNN约50%(甚至更少)的计算时间，以此优化了SNN在冯·诺伊曼结构下计算开销巨大的问题。

    3. 讨论了好奇相关的多脑区、多认知功能协同交互学习系统

    从基于好奇的计算系统而言，本文分析了与好奇相关的初级视觉皮层信息处理和高级认知脑区之间的协同关系。重点讨论了自上而下的神经递质和信号传递对初级视觉皮层的全局和局部调控机制(有助于深度脉冲网络的优化)，以及类好奇的主动学习多脑区协同架构(有助于智能体实现时空信息处理、异常场景感知、快速强化学习以及一定程度的迁移学习能力)。

Traditional Artificial Neural Networks (ANNs) have shown excellent performance in multiple pattern recognition tasks, such as the computer vision and natural language processing. But due to the dependence on a large number of training samples and powerful hardware, and their weak incremental learning and adaptive abilities, some computational models based on Deep Neural Network (DNN) still remain a lot of space for improvement. In recent years, Spiking Neural Network (SNN) has attracted a lot of attention with its more biologically plausible structures and the signal computation. And because it can be more integrated with the hierarchical structures and multiple plasticity rules of mammalian brain, SNN is considered to have more potential to break through the obstacles of insufficient robustness and huge computational cost of current artificial intelligence models. At the same time, mechanisms in the brain like curiosity and their cognitive pathways involved play a crucial role in our active and efficient lifelong learning. Therefore, this paper will base on the optimization of SNN, and explore the way of combination between curiosity and SNN. The main points are as follows:

   1. Proposed the optimization of SNN inspired by multiple biological rules

      From the point view of SNN learning algorithms, different from transforming the activation function of SNN into a continuous differentiable form similar to that in ANN, or converting the original spike train into firing rate, we directly modeled the mechanism of equilibrium membrane potential in brain and proposed the Voltage-driven Plasticity-centric SNN (VPSNN), which could achieve the accuracy of 98.52% on MNIST dataset and this is the state-of-the-art result under pure biological plasticities.

   2. Built the curiosity-based Spiking Neural Network

      From the point view of SNN efficiency improvement, inspired by the characteristics and advantages of curiosity mechanism in the brain, this paper proposed an effective method to integrate current SNN learning rules with it, i.e. the Curiosity-based Spiking Neural Network (CBSNN). On the premise of comparable accuracy, CBSNN saves around 50%(even more) computation cost than VPSNN on MNIST, Iris, NETtalk, Fashion-MNIST and CIFAR-10, which improves the issue of huge computation cost of SNN under the von Neumann framework.

    3. Discussed the curiosity-based multi-brain and multi-cognitive interactive learning system

      From the point view of computing systems based on curiosity, this paper analyzed the cooperation between primary visual cortex information processing and higher cognitive brain regions which are related to curiosity, i.e. the global and local regulation mechanisms of the primary visual cortex by top-down neurotransmitters and signals which will contribute to the optimization of deep spiking neural networks, and the active learning based on multi-brain collaborative architecture which will help agents to realize spatial and temporal information processing, abnormal scene perception, quick reinforcement learning and multi-task transfer learning.