类脑智能研究中心知识库

类脑自主学习与决策模型旨在从多个尺度建模生物脑多脑区协同的自主决策网络结构和工作机制。生物脑决策系统包括快决策和慢决策两种，其中快决策是无意识的本能决策，例如生物在自然界中本能地躲避天敌，而慢决策是有意识地经过分析得到的决策，通过与环境交互不断更新策略的自主学习就是一种慢决策。神经科学的发展提供了不同尺度学习和决策等认知功能的神经机制，包括微观分子尺度的动态变化，以及宏观尺度的多脑区协同神经环路，这些发现都有助于推动人工智能朝着更加类人的方向发展。 受不同生物脑自主学习与决策的神经机制启发，本文从多个尺度构建类脑自主学习与决策模型。从微观尺度来讲，本文对发育过程中神经元、突触和树突棘的生长与消亡机制进行分析建模，并启发自适应的动态神经网络。从宏观尺度来讲，受不同生物自主决策神经机制的启发，建模多脑区协同的自主决策神经网络模型，并应用在无人机上完成自主决策任务，包括无人机自主飞过窗、飞过门和自主避障。

本文的主要工作和创新点归纳如下：

第一，类哺乳动物多脑区协同的自主决策模型。借鉴哺乳动物脑的自主决策神经环路，提出了两种受脑启发的自主决策模型。具体而言，提出了一种基于前额叶皮层对基底神经节指导的类脑自主决策模型，通过连续奖励函数和相对奖励函数实现了基于少量状态下的快速决策；使用脉冲神经网络建模了哺乳动物多脑区协同的自主决策神经环路，分析了不同脑区在自主决策过程中各自所起的不同作用，并应用在无人机自主飞过窗、过门和自主避障任务上。与传统强化学习算法相比，该模型更具有生物可解释性，网络中的每个脑区都有着不可替代的作用，多个脑区协同起来实现类生物智能的认知行为。在真实场景下的无人机自主决策任务上，所提算法只需要少量状态空间就可以快速地完成任务，实现了基于小样本的在线自主学习。

第二，类果蝇脑自主决策脉冲神经网络模型。借鉴果蝇脑中的线性和非线性通路，其中线性通路是基于简单感知的快速决策通路，非线性通路是基于效价的两难决策通路，果蝇能够根据任务的需求自适应地在两条通路之间切换。本文使用脉冲神经网络建模果蝇线性和非线性决策的神经环路，得到了和生物实验一致的结论，在无人机强化学习和反转学习任务上均得到了有效的验证。果蝇的非线性通路能够在两难抉择情况下做出清晰的决策，将非线性通路机制应用在无人机真实场景下的多难决策任务上，可以帮助无人机快速地选择收益最大的行为，进而帮助加快决策。

第三，基于视觉恐惧反应通路的快速决策模型。受脑视觉恐惧反应通路的信息处理机制启发，建模了视网膜-上丘-丘脑枕-杏仁核的视觉恐惧反应环路，实现了对于快速投射物体的逼近式运动检测，以及无人机本能躲闪抱枕、拳头等投射物。视觉恐惧反应通路是在数亿年进化过程中高度保留的用于本能防卫的第二视觉通路，这是一条本能的无意识的快决策通路，与第一视觉通路的双目匹配算法对比，所提模型能够更快地检测到危险投射物。

第四，基于脑发育机制的自适应神经网络。宏观尺度的多脑区协同神经环路离不开微观尺度对网络的动态调整。本文进一步从微观尺度考虑大脑发育机制对神经网络的动态修饰作用，提出了融合神经元、突触和树突棘的动态变化机制的自适应神经网络模型。具体来说，引入神经元消亡机制可以动态地优化网络的结构以适应不同复杂度的任务，实现了神经元的动态分配；引入突触剪枝机制可以去除冗余的突触连接，有效地降低网络的复杂度，避免过拟合并提升网络的收敛速度；引入树突棘的形成与修剪机制可以动态地限制不必要突触的生长，同时促进有用的突触连接，进而显著地提升神经网络的性能和收敛速度。大量实验验证了从微观尺度借鉴脑发育过程的动态变化机制可以提升网络的适应性、帮助网络更加高效准确的工作。

Brain-inspired learning and decision-making neural network model simulates the multi-scales and multi-brain areas coordinated network architecture and working mechanism of brain learning and decision-making through computational modeling, and applies it to decision-making tasks on intelligent agents. Brain decision-making system contains fast decision-making and slow decision-making, in which the fast decision is unconscious and instinctive, such as creatures instinctively evade natural enemies in nature, while slow decision-making is conscious and analytically determined decision. Autonomous learning that continuously updates strategies through interacting with the environment is a kind of slow decision-making. The development of neuroscience provides neural mechanisms for learning and decision-making at different scales, including dynamic changes in microscopic molecular scales, as well as multi-brain coordinated neural circuits on macro scale, all of which contribute to the advancement of artificial intelligence toward humanoid information processing mechanism. Inspired by the neural mechanisms of brain learning and decision-making, this paper builds brain-inspired autonomous learning and decision-making models from multiple scales. At the microscopic scale, the growth and extinction mechanisms of neurons, synapses and dendritic spines during development are analyzed and modeled, and inspiring adaptive dynamic neural networks. At the macroscopic scale, inspired by the decision-making systems of different biological brain, the multi-brain area coordinated decision-making neural network model is modeled and applied to the UAV to perform autonomous decision-making tasks, including the UAV flying through window, door and the UAV obstacle avoidance.

The main contributions are summarized as follows:

1) Mammalian-inspired multi-brain areas coordinated decision-making model. Based on the autonomous decision-making neural circuit of the mammalian brain, two brain-inspired autonomous decision-making models are proposed. Specifically, a brain-inspired decision-making model based on top-down biasing of prefrontal cortex to basal ganglia is proposed. Through continuous reward function and relative reward, rapid decision based on a small number of states is realized. A brain-inspired decision-making spiking neural network (BDM-SNN) is proposed to model the mammalian multi-brain areas coordinated decision-making neural circuits. Then we analyze the role of different brain areas in the process of autonomous decision-making. Applications on the Unmanned Aerial Vehicle (UAV) flying through window, door and the UAV obstacle avoidance tasks demonstrate the effectiveness of the proposed model. Compared with traditional reinforcement learning and existing biologically inspired methods, our method contains more biologically-inspired mechanistic principles, and each brain area has an irreplaceable role. Multiple brain areas work together to realize cognitive behaviors like bio-intelligence. On the UAV autonomous decision-making task of the real scene, the proposed model only needs a small state space to complete the task quickly, and can realize online learning based on small samples.

2) Drosophila-inspired decision-making spiking neural network model. Inspired by the fast decision and slow decision paths (linear and nonlinear pathways) in the Drosophila, where the linear pathway is a fast decision-making pathway based on simple perception, the nonlinear pathway is value-based decision, and the decision making process can freely switch between two distinct neural circuits based on the demand of visual learning task. We propose a general computational spiking neural network (SNN) model inspired by Drosophila to explore how different brain areas connected together contributing to linear and nonlinear decision-making behavior. First, our SNN model could successfully describe all the experimental findings in fly visual reinforcement learning and action selection behavior among multiple conflicting choices as well. Second, our computational modeling show that DA-GABA-MB works in a recurrent loop providing a key circuit for gain and gating mechanism of nonlinear decision making. Finally, based on the proposed model, the unmanned aerial vehicles (UAV) could quickly learn to make clear-cut decision among multiple visual choices and flexible reversal learning.

3) Fast decision model based on visual fear response pathway. Inspired by the information processing mechanism of the brain visual fear response pathway, we model the retina-superior colliculus-pulvinar-amygdala loop, and realize the looming motion detection for fast projecting objects. The proposed model is applied to the UAV instinct dodging pillow and fist projections. The brain visual fear response pathway is the second visual pathway for instinct defense that is highly preserved during hundreds of millions of years of evolution. This is an instinctive unconscious fast decision pathway, compared with the binocular matching algorithm of the first visual pathway, the proposed model can detect dangerous projectiles more quickly and accurately.

4) Adaptive neural network based on brain development mechanism. Inspired by the dynamic modification of neural network by microscopic scale brain development mechanism, neural networks combining dynamic changes of neurons, synapses and dendritic spines are proposed. Specifically, the introduction of neuron extinction mechanism can dynamically optimize the structure of network to adapt to tasks with different complexity, and realize the dynamic allocation of neurons; the introduction of synaptic pruning mechanism can remove redundant synaptic connections and effectively reduce the complexity of the network in order to avoid over-fitting and improve the convergence speed; the introduction of dendritic spine formation and pruning mechanisms can dynamically limit unnecessary synapses while promote useful synaptic connections, thereby significantly improving the performance and convergence speed of the neural network. Extensive experiments indicate that the dynamic change mechanism of microscale brain development can improve the adaptability of the network and help the network works more efficiently and accurately.