中国科学院自动化研究所机构知识库

目前绝大多数的神经网络结构都不同程度受到了经典皮层解剖学的启发，通过逐层抽象的方式，对信息进行层次化处理。脑皮层是人脑高级认知功能的汇集区域，多模态感知、学习、决策、推理等认知功能都依赖于脑皮层信息处理机制。脉冲神经网络作为第三代神经网络，以离散的脉冲传递信息，更加类生物，在时间及空间信息的处理能力上更加强大，结合脑皮层信息处理机制，能更好地构建功能强大的脉冲神经网络。脑皮层完成各项认知功能离不开丘脑的集成、协同以及自组织控制。丘脑作为一簇前脑核团，接收除了嗅觉之外的多种感觉信息，并传递到大脑皮层，在皮层区域之间整合传递信息。皮层及丘脑协同计算是实现认知功能模拟的关键。
受皮层丘脑环路启发，本文通过皮层区域宏观连接的分析，构建了受跨脑区反馈学习机制启发的脉冲神经网络模型，受脑区内反馈以及兴奋抑制性神经元平衡启发的脉冲神经网络模型，通过对于皮层丘脑双向连接的分析，从微观突触优化的角度构建了受皮层丘脑环路启发的视觉分类模型，从宏观决策角度构建了受皮层丘脑环路启发的多视觉区协同跟踪模型。本文的主要工作和创新点归纳如下：
第一，本文构建了受跨脑区反馈学习机制启发的脉冲神经网络模型。人脑中存在着大量的反馈连接，反馈连接将高级皮层的全局信息以自顶向下的方式传递到低级皮层，受此启发，本文构建了一个随机的反馈连接帮助脉冲神经网络将输出层的误差直接直接传递到之前层，同时，利用一个微分脉冲时序依赖可塑性原理来优化局部的突触可塑性。在MNIST以及Fashion MNIST上的实验结果表明，此算法达到了由反向传播算法训练的脉冲神经网络的性能。	
第二，本文构建了受脑区内反馈以及兴奋抑制性神经元平衡启发的脉冲神经网络模型。受大脑中自突触连接以自反馈形式连接胞体的启发，构建了一个自适应的延时自反馈机制作用在膜电势上来调节脉冲发放的精度；并且采用兴奋抑制性神经元平衡的机制来动态控制脉冲神经元的输出。在基于反向传播训练的脉冲神经网络上引入这两个机制后，在多个数据集上的实验结果表明，这两个机制不仅提升了网络收敛的速度，还提升了网络的性能。在MNIST、Fashion MNIST、N-MNIST数据集上，本文提出的算法达到了目前所知的最优的性能，在CIFAR10数据集上，和目前最好的脉冲神经网络相比，此网络以一个轻量级的结构达到了较好的性能。
第三，受皮层丘脑环路启发的视觉分类模型。从解剖学角度看，皮层区域之间与丘脑存在着大量的双向连接，多个特殊的丘脑核团将初级皮层区域和高级皮层区域连接在一起，许多理论认为皮层-丘脑-皮层的这种交互对于全局信息的处理至关重要。受丘脑核团和不同皮层区域间的网络拓扑结构启发，本文利用一个多层的神经网络模拟皮层，利用额外的无监督赫布学习来模拟丘脑对于不同皮层区域之间的调控。结果表明，此网络在较小的训练数据集上性能高于基于皮层的网络，且对于学习率、激活函数、网络结构等超参的鲁棒性较高。	
第四，受皮层丘脑环路启发的多视觉区协同跟踪模型。由于卷积神经网络强大的特征表示能力，结合卷积神经网络和相关滤波在视觉跟踪任务中得到了广泛应用。卷积神经网络的深层包含更多的语义信息，浅层包含更多的细节信息，受丘脑对于视皮层决策的动态协同作用的启发，本文利用空间注意机制选择出了最重要的一层作为基础响应，其余的响应为辅助响应；利用时间注意机制确定每个辅助响应的权重。最后，根据融合响应的最大值定位目标。本文进一步将相关滤波以及重要性分配机制融入到卷积神经网络中，构建了一个端到端的跟踪算法，在OTB-2013数据集达到了与目前已知工作相比最优的性能，在OTB-2015数据集上达到了较为优异的性能。

At present, most of the neural network structures are inspired by the classic cortical anatomy to varying degrees, and the information is processed hierarchically through layer-by-layer abstraction. The cerebral cortex is the gathering area of the human brain’s advanced cognitive functions. Cognitive functions such as multimodal perception, learning, decision-making and reasoning all rely on the information processing mechanisms of the cerebral cortex. As the third generation of neural network, the spiking neural network transmits information with discrete spikes, which is more biological and more powerful in the processing of temporal and spatial information. Combined with the cortical information processing mechanism, it can better build a powerful spiking neural network. The completion of various cognitive functions of the cerebral cortex is inseparable from the integration, coordination and self-organizing control of the thalamus. As a cluster of forebrain nuclei, the thalamus receives multiple sensory information except for olfaction, transmits them to the cerebral cortex, and integrates and transmits information between cortical areas. The coordinated computing of the cortex and thalamus is the key to realize the simulation of cognitive function. 
Inspired by the cortical-thalamic pathway, this thesis analyses the macroscopic connections of the cortex to construct a spiking neural network that is affected by the cross-area feedback connections and local synaptic plasticity, and a spiking neural network that is affected by inner-area feedback connections and the balance of excitatory inhibitory neurons. For the analysis of the bidirectional connections of the cortex and thalamus, a visual classification model inspired by cortical-thalamic pathway is constructed from the micro synaptic optimization, and  a multi-cortical collaboration tracking model inspired by cortical-thalamic pathway is constructed from the macro decision-making perspective. The main work and innovations of this thesis are summarized as follows:
First, this thesis constructs a spiking neural network that is affected by the cross-area feedback connections and local synaptic plasticity. There are a large amount of cross-area feedback connections in the brain. The feedback connections transfer the global information from the higher cortex area to the lower cortex area in a top-down manner. Inspired by this, this thesis build random feedback connections to help the spiking neural network transfer the error from the output layer directly to the previous layers. A differential spiking time dependent plasticity is used to optimize the local synapses. Extensive experimental results on the benchmark MNIST and Fashion MNIST have shown that the proposed algorithm performs favorably against several state-of-the-art SNNs trained with backpropagation.
Second, this thesis constructs a spiking neural network that is affected by inner-area feedback connections and the balance of excitatory inhibitory neurons. Taking inspiration from the autapse in the brain which connects the spiking neurons with a self-feedback connection, this thesis applies an adaptive time-delayed self-feedback on the membrane potential to regulate the spike precisions. As well as, this thesis applies the balanced excitatory and inhibitory neurons mechanism to control the spiking neurons' output dynamically. With the combination of the two mechanisms into BP-based spiking neural networks, the experimental results on several standard datasets have shown that the two modules not only accelerate the convergence of the network but also improve the accuracy. For the MNIST, Fashion MNIST, and N-MNIST datasets, this model has achieved current known state-of-the-art performance. For the CIFAR10 dataset, this model also gets remarkable performance on a relatively light structure that competes against state-of-the-art SNNs.
Third, this thesis constructs a visual classification model inspired by cortical-thalamic pathway. From the anatomical point of view, there are a large number of bidirectional connections between the cortex and the thalamus. Multiple special thalamic nuclei connect the primary cortex and the higher cortex. Numerous theories suggest that the cortico-thalamo-cortical communication is crucial for global information processing. Inspired by the network topology between thalamic nuclei and different cortical areas, this thesis uses a multi-layer neural network to simulate the cortex and use additional unsupervised Hebbian learning to simulate the regulation of the thalamus on different cortex areas. The results show that, the performance under the small training dataset is higher than the network based on cortex, and it is more robust to hyperparameters such as learning rate, activation function, and the network structure. 
Forth, this thesis constructs a multi-cortical collaboration tracking model inspired by cortical-thalamic pathway. Due to the powerful feature representation capabilities of convolutional neural networks, the combination of convolutional neural networks and correlation filters has been widely used in visual tracking. Since the deep layer of the convolutional neural network contains more semantic information, and the shallow layer contains more detailed information, due to the collaboration of the thalamus on the decision-making of the visual cortex, the most import layer is selected as the basic response by using the spatial attention mechanism. The remaining responses are auxiliary responses, the temporal attention mechanism is used to determine the weight of each auxiliary response. Finally, the target is located by the maximum value of the fused responses. Furthermore, the correlation filter and dynamic importance alignment module are integrated into the convolutional neural network, and an end-to-end tracking algorithm is constructed, which achieves the current known state-of-the-art performance on the OTB-2013 dataset and comparable performance on the OTB-2015 dataset.