中国科学院自动化研究所机构知识库

基于类脑脉冲神经网络的音乐学习旨在从多个尺度借鉴大脑在音乐处理方面的工作机制，建立一个类生物的多脑区协同计算模型，从而实现智能计算模型对音乐的学习与创作。有关音乐的研究涉及多个方面，例如记忆、认知、运动、情绪和心理等等。无论在神经科学领域还是在人工智能领域，音乐学习都是重要的研究课题。神经科学领域的相关研究表明，音乐是一种复杂且具有时间特性的声音刺激，人脑中多个功能区域会参与音乐信息的处理。同时，神经科学研究者们也从多个尺度上研究了相关脑区的结构功能及其之间的协同机制。神经科学领域在音乐研究方面的突出贡献促使我们研究新的人工智能方法去理解和应用与音乐有关的科学问题。

受大脑在音乐处理方面神经机制的启发，本文构建了一个多尺度的类脑脉冲神经网络模型，主要研究与探讨音乐的感知、记忆、知识的表征与学习以及音乐创作几个问题。在微观层面上，本文借鉴生物神经元的放电特性，分别为兴奋性神经元和抑制性神经元建模，并利用生物大脑中突触的可塑性神经机制来调节神经网络。在介观层面上，本文参考了大脑在音乐处理时涉及的脑区及脑区间的环路，模拟了多个脑区及其之间的环路，形成了一个多脑区协同的神经网络，通过激活不同的脑区及其环路，实现了音乐的感知记忆与学习等多项任务。在宏观层面上，本文参考人类在音乐方面的行为活动，实现了音乐回忆、节奏调控和作曲等类人认知任务。基于以上研究，本文的创新点总结如下：

第一，通过借鉴大脑在音乐感知与记忆方面的神经机制，构建了一个类脑脉冲神经网络的音乐记忆模型。具体而言，本文深入调研了大脑听觉系统对音乐的感知机制以及记忆机制，提出一种基于类脑脉冲神经网络的音乐记忆模型，能够对音乐曲目进行感知编码、记忆与回忆。模型首先借鉴了大脑听觉系统对音符的处理机制，分别对音符的音高以及时长进行编码；其次，受大脑记忆机制的启发，利用突触可塑性学习原理，模型能够对乐曲的音符进行有序学习与存储；此外，模型还实现了两种类人行为模式的音乐回忆，即根据乐曲名回忆乐曲以及根据乐曲片段回忆乐曲。目前根据我们调研的结果，该模型首次将类脑脉冲神经网络用于音乐记忆之上，实现了多个与记忆相关的任务，并具有较高的精确度。与传统算法相比，该模型也具有更强的生物可解释性。

第二，通过分析与借鉴大脑韵律感知与调控的神经机制，构建了一个类纹状体的脉冲神经网络，实现了大脑在回忆乐曲时的速度调控。具体而言，该模型主要模拟了纹状体中调节韵律的神经元群体，并与记忆网络进行连接；通过调整该类纹状体网络神经元发放脉冲的频率，改变记忆网络中神经元的活动，从而控制回忆乐曲时的韵律与速度。目前根据本文的调研结果，该模型是首次将脉冲神经网络用于音乐韵律调整问题之上，实现了与记忆网络的相互协同，并能够有效的调控网络节奏，在一定程度上实现了行为类人的准则。

第三，受大脑在进行音乐创作时相关的脑区结构及工作机制的启发，本文构建了一个类脑脉冲神经网络的多风格音乐作曲模型。具体而言，本文构建了一个类脑脉冲神经网络的层次化知识学习网络，在音乐记忆的模型基础上，引用生物神经元模型以及神经元动态生长机制，实现了音乐相关的理论与知识的编码与存储；利用突触可塑性原理以及神经震荡机制，使得知识网络与记忆网络能够动态地进行协同与学习，完成有序音符与音乐知识的联想学习；模型在以上记忆网络以及层次化的知识网络基础上，可根据古典音乐的流派以及不同音乐家的个人特色，创作相应风格的乐曲。通过实验结果及用户评估，该模型生成的大多数旋律都分别具有各流派或各音乐家的特色。目前根据我们调研的结果，该模型是首次将脉冲神经网络用于音乐作曲之上，并实现了不同古典流派，不同音乐家风格的作曲任务。

Inspired by neuroscientific mechanisms of the brain in music processing from multiple scales, the paper aims to establish a biological-inspired and multi-brain areas coordinated model which is based on a spiking neural network to process the problems of music learning. The study involves many aspects, such as memory, cognition, movement, emotion, psychology and so on. Both in neuroscience and artificial intelligence, music is an important research topic. Related studies in the neuroscience domain have shown that music is a complex and temporal acoustic stimulus, and multiple functional areas of the human brain are involved in musical information processing. Furthermore, neuroscientists have also studied the structures and functions of related brain regions and their coordinated mechanisms at multiple scales. The outstanding contribution of the neuroscience field to music research has led us to develop new artificial intelligence methods to study related problems of music learning.

Inspired by the brain's related neural mechanisms in music processing, this paper constructs a multi-scale brain-inspired spiking neural network, mainly researching and discussing music perception, memory, knowledge representation and learning, and music creation. At the micro-level, this article draws on the firing characteristics of biological neurons, simulates excitatory neurons and inhibitory neurons, and uses the synaptic plasticity in the biological brain to regulate neural networks. At the mesoscopic level, this article refers to the effective circuits of brain regions involved in music processing in the brain, and builds a multi-brain region coordinated network. Here, each brain region performs its own duties to complete perceptual memory and learning, creation and other tasks. Based on the above description, the innovations of this article are summarized as follows:

First, inspired by the brain's neural mechanisms in music perception and memory, a brain-inspired spiking neural network model of music memory is constructed. Specifically, this article deeply investigates the perception and memory mechanism of the brain's auditory system, and proposes a music memory model based on a spiking neural network that can perceive, encode, memorize and recall music tracks. The model draws on the auditory system's processing mechanism for notes, and encodes the pitch and duration of the notes respectively. Inspired by the brain’s memory mechanism, the model uses the synaptic plasticity learning rules to store the pitches and durations of ordered notes. In addition, the model also implements two modes of musical retrieving, namely, music retrieving based on its name and retrieving based on musical episodes. According to our best knowledge of investigation, this model is the first attempt to use the brain-inspired spiking neural network for music memory, achieving multiple memory-related tasks with high accuracy. Compared with traditional algorithms, this model also has stronger bio-interpretability.

Secondly, inspired by the neural mechanism of the brain in the perception and modulation of rhythm, a striatum-inspired spiking neural network is constructed to adjust the speed of the brain in the musical retrieval process. Specifically, the model mainly simulats the neural populations in the striatum that can modulate rhythm, and connects them to the memory subnetwork. By adjusting the frequency of the neural activities in the striatum network, the spiking rates of the neurons in the memory network are regulated, so as to control the rhythm and speed of the retrieving process. At present, according to the results of our investigation, this model is the first attempt to apply the spiking neural network to the adjustment of music retrieving speeds. 

Thirdly, inspired by the structure and working mechanism of related brain regions in the process of music creation, a multi-style music composition model based on brain-inspired spiking neural network is constructed. Specifically, this paper first constructs a hierarchical knowledge learning model of spiking neural network. Based on the music memory model, the rules of neural dynamic growth mechanism are used to realize the encoding and storing the musical theories and knowledge. Based on the synaptic plasticity learning rules and neural oscillations, the knowledge network and memory network can dynamically cooperate with each other, so as to complete the associative learning of ordered notes and musical knowledge. Based on the above memory network and hierarchical knowledge network, the model can create musical melodies with the corresponding style according to the genre or individual characteristics of different musicians. The experimental results show that most of the melodies generated by this model have the genre or musician characteristics. At present, according to the results of our investigation, this model is the first attempt to apply the spiking neural network to music composition, and realize the generation of music with different classical genres and different musician styles.