毕业生知识库

受生物脑脉冲信息传递与处理机制启发，类脑脉冲神经网络建模生物神经元的动态计算与脉冲输出过程，相比于传统人工神经网络， 具有更高的能效比和生物合理性。脉冲神经网络在信息编码、结构特征与优化应用等众多方面都有区别于传统人工神经网络的独特之处。神经科学研究表明，神经元脉冲在频率和相位两个维度编码信息，拓展了大脑信息表征能力。同时，神经元的结构特征也在大脑信息处理过程中起到至关重要的作用。神经元树突、胞体等不同部位具有相对独立的信息计算能力，单个神经元不同部位计算效果的组合产生复杂的动力学特性。脉冲神经网络在信息编码和结构特征方面的计算建模，奠定了类脑学习模型处理复杂感认知问题的计算基础。

      本文从信息编码、学习模型和认知应用三个方面研究类脑脉冲神经网络。在信息编码方面，构建频率-相位相结合的脉冲序列时空维度编码方法；在模型结构方面，受大脑皮层神经元结构特性启发，构建多房室脉冲神经元模型，在神经元模型中分别建立负责处理不同信息的树突和胞体结构，提高单个神经元处理复杂感认知信息的能力。同时，本文将脉冲序列编码和模型结构结合起来，构建多房室脉冲神经网络，实现深度脉冲网络在复杂感知、决策场景上的应用。本文的主要工作与创新点归纳如下：

       一、受量子叠加态启发的脉冲序列时空维度编码方法。传统人工神经网络使用标量实值表征信息，而脉冲序列的频率和相位特征维度拓展了其信息编码能力。目前脉冲神经网络模型往往只采用频率或者相位单个维度编码脉冲信息，而生物神经元脉冲序列同时使用频率和相位两个维度编码信息，具有高维时空信息表示能力。本文使用量子计算的数学框架，借鉴量子图像处理方式，提出量子叠加态时空脉冲序列编码方法，将脉冲序列的频率和相位特征结合起来，映射到表征量子态的复数希尔伯特空间，实现频率和相位分别编码不同信息特征。本文将量子叠加态脉冲编码应用到处理背景翻转的图像。实验证明，相比只使用频率或相位编码方法，使用脉冲序列频率和相位相结合的时空维度编码方法能取得更强的适应图像背景变换和噪声干扰的能力。

       二、多房室脉冲神经网络模型与优化算法。受大脑皮层锥体神经元结构与信息处理机制启发，本文构建类脑多房室神经元模型，将神经元的树突和胞体视为独立的信息接收与处理单元，使得单个神经元具有不同的信息计算模块，基于此构建用于模式识别任务的多房室脉冲神经网络。本文将多房室脉冲神经网络与量子叠加态时空脉冲序列编码方法结合，用于处理背景翻转和噪声图像。实验证明，在原始图像数据上训练，传统卷积神经网络无法识别背景翻转后图像。与此相比，得益于脉冲序列时空表征维度拓展和多房室神经网络复杂动态计算过程，本文提出的脉冲神经网络能够在图像背景翻转后保持模型性能，在加入噪声的图像数据上也表现出超过传统卷积神经网络的鲁棒性。

       三、膜电势正则化深度脉冲神经网络强化学习模型。目前脉冲神经网络研究与应用主要集中在图像分类、目标检测与跟踪等视觉监督学习任务。脉冲神经网络在复杂决策任务（比如深度强化学习）上的应用还比较少。尽管之前有关于脉冲神经网络和强化学习结合的研究，但大多数集中在浅层网络机器人控制问题，或者使用ANN-SNN转换方法来解决深层网络的权重调节问题。 本文从数学分析深层脉冲神经网络中脉冲序列传递特征消失问题，借鉴神经元膜电势受局部场效应影响的机制，提出了一种层级膜电势正则化方法，实现了深度脉冲神经网络在视觉输入的强化学习任务上的应用。实验表明，与ANN-SNN转换方法和其他脉冲神经网络强化学习工作相比，本文所提出的膜电势正则化深度脉冲状态-动作值网络在Atari游戏任务上取得了目前最好的性能。

       四、类脑多房室深度脉冲神经网络值分布强化学习模型。认知神经科学研究表明，大脑采用概率分布来预测决策与推理过程所有可能奖励。本文结合多房室脉冲神经网络模型和群体神经元脉冲序列编码方法，构建深度脉冲神经网络值分布强化学习模型。多房室神经网络的树突接受并处理不同来源的决策信息，并在胞体部分进行信息整合产生不同状态-动作值的概率分布估计。神经元群体编码将分位数连续实数值表征到可分的脉冲序列空间，实现深度脉冲神经网络分位数回归算法。实验证明，本文提出的多房室脉冲神经网络值分布强化学习模型在Atari任务上的性能表现超过了FQF等传统基于人工神经网络的深度值分布学习算法。

Inspired by the transmission and processing mechanism of spike information in the biological brain, the brain-inspired spiking neural network models the dynamic calculation and binary spiking output process of biological neurons. Compared with traditional artificial neural networks, it has better energy efficiency and biological plausibility. The spiking neural network is different from the traditional artificial neural network in many aspects, such as information encoding, structural characteristics, optimization and applications. Biological studies have shown that neuron spike encode information in two dimensions of frequency and phase, expanding the ability of the brain to represent information. At the same time, the structural characteristics of neurons also play a crucial role in the process of brain information processing. Different parts such as dendrites and soma of  neurons have relatively independent information computing capabilities, and the combination of computing effects of different parts of a single neuron produces complex dynamic characteristics. The model construction of the spiking neural network in terms of information encoding and structural features has laid the computational foundation for brain-like agents to deal with complex perception and cognition problems.

     In this paper, the brain-inspired spiking neural network is studied from three aspects: information encoding, model structure and application. In terms of information encoding, a spiking sequence spatio-temporal dimension encoding method combining frequency and phase is constructed; in terms of model structure, inspired by the connection of cerebral cortex neurons, a multi-compartmental spiking neuron model is constructed, in which dendrites and the soma undertakes different information computing function, improving the ability of a single neuron to process complex perception and cognitive information. At the same time, this paper combines the spike sequence encoding and model structure to construct a multi-compartment spiking neural network to realize the application of deep spiking networks in complex perception and decision-making scenarios. The main work and innovation of this paper are summarized as follows:

    First, quantum superposiiton inspired spike sequence encoding method. The traditional artificial neural networks use scalar real values to represent information, while the frequency and phase feature dimensions of spiking sequences expand their information encoding capabilities. Current spiking neural network models often only use frequency or phase single-dimensional encoding method. Using the mathematical framework of quantum computing and referring to quantum image processing methods, this paper proposes a quantum superposition spatio-temporal spike sequence encoding method, which combines the frequency and phase characteristics of the spike sequence and maps it to the complex Hilbert space that characterizes the quantum state. The frequency and phase are used to encode different information features respectively. In this paper, quantum superposition state spike coding is applied to processing images with reverse background. Experiments show that compared with only frequency or phase encoding methods, the spatio-temporal spike sequence encoding method using  can obtain better ability to deal with background reverse and noise added image data.

    Second, multi-compartment spiking neural network model and optimization algorithm. Inspired by the structure and information processing mechanism of pyramidal neurons in the cerebral cortex, this paper constructs a brain-inspired multi-compartmental neuron model, and regards the dendrites and soma of neurons as independent information receiving and processing units, so that a single neuron has different information computing part, and construct a multicompartmental spiking neural network for pattern recognition tasks. In this paper, a multi-compartment spiking neural network is combined with a quantum superposition spatio-temporal spike sequence encoding method to process reverse background and noise added images. Experiments have proved that the traditional convolutional neural network cannot recognize the image after the background is reversed if it is only trained on the original image data. In contrast, the spiking neural network proposed in this paper can maintain the model performance after the image background is reversed, due to the dimension expansion of the spatio-temporal representation of the spike sequence and the complex dynamic calculation process of the multi-compartmental neural network. At the same time, the model in this paper also shows more robustness than the traditional convolutional neural network on the image data with noise.

    Third, deep spiking neural network reinforcement learning model with membrane potential based normalization. At present, the research and application of spiking neural network mainly focus on visual supervised learning tasks such as image classification, object detection and tracking. The application of spiking neural networks to complex decision-making tasks (such as deep reinforcement learning) still need to be explored. Although there are previous studies on the combination of spiking neural networks and reinforcement learning, most of them focus on robot control problems with shallow networks, or use ANN-SNN conversion methods to implement weight adjustment problems with deep networks. In this paper, the problem of the disappearance of the transmission features of the spike sequence in the deep spiking neural network is analyzed mathematically. And inspired by the mechanism that the neuron membrane potential is affected by the local field effect, this paper proposes a membrane potential based layer normalization method and successfully applies the deep spiking neural network to visual input reinforcement learning tasks. Experiments show that compared with the ANN-SNN conversion method and other spiking neural network reinforcement learning works, the proposed spiking deep Q network achieves the best performance on the Atari game task.

    Finally, brain-inspired multi-compartment spiking deep distributional reinforcement learning model. Cognitive neuroscience research shows that the brain uses probability distributions to represent different states and action selection rewards in the decision-making and reasoning process. In this paper, a spiking deep distributional reinforcement learning model is constructed by combining the multi-compartment spiking neural network model and the population neuron spike sequence encoding method. The dendrites of the multi-compartment neural network receive and process decision-making information from different sources, and integrate information in the soma compartment to generate probability distribution estimates for different state-action values. Neuron population encoding method represents quantile continuous real values into separable spike sequence space, and realizes quantile regression algorithm in deep spiking neural network. Experiments show that the multi-compartment spiking deep distributional reinforcement learning model proposed in this paper achieve better performance than traditional deep distributional learning algorithms based on artificial neural networks on Atari tasks.