CASIA OpenIR  > 类脑智能研究中心  > 神经计算及脑机交互
脑电情绪识别中跨被试迁移学习方法研究
李劲鹏
Subtype博士
Thesis Advisor何晖光
2019-05
Degree Grantor中国科学院自动化研究所
Place of Conferral中国科学院自动化研究所
Degree Discipline模式识别与智能系统
Keyword脑-机接口,情绪识别,深度学习,迁移学习,领域自适应,对抗训练
Abstract

情绪识别有助于构建更加友好的人机交互系统,并在安防领域及多种精神疾病的诊断、治疗和康复中扮演重要角色。与基于表情、动作和语言的传统方法相比,基于脑电(Electroencephalogram,EEG)的情绪识别更加稳定和可靠。然而,不同被试的EEG数据分布差异大,通用模型的泛化能力有限,情绪识别模型往往需要针对每个被试分别获取,这极大限制了情绪识别的便利性。

本文从机器学习的角度出发,试图解决EEG情绪识别模型的跨被试适配问题,即在新被试缺乏带标签样本的情况下,如何结合来自过往被试的信息获得可靠的情绪识别模型。根据新被试数据标注情况的不同,我们分别提出了监督式、半监督式和无监督式迁移学习策略。论文的主要工作和创新点归纳如下:

1. 基于深度学习的高精度的情绪识别

传统做法一般是将多通道EEG数据中提取到的特征拼接为特征向量,再使用机器学习方法建立特征向量到情绪标签的映射。然而,这种方法并没有直接考虑电极之间的相互关系。为了使模型纳入和使用更多有用的信息,我们设计了一个保留电极摆放拓扑关系的卷积神经网络,其浅层在多个位置上挖掘相邻电极的局部响应模式,深层则负责综合全脑的情绪相关特征。该方法兼顾了电极的局部响应模式和全局响应模式。结果表明,我们提出的方法优于传统的方法,可基于低信噪比的EEG数据实现高精度的分类。在上述工作的基础上,我们使用新被试的少量带标签样本,通过神经网络微调的方法实现模型的跨被试适配。

2. 提出了一种多源迁移学习框架

我们将新被试的脑电分布视为目标域,所有过往被试的数据都可以作为知识源。针对目标域中有少量带标签样本,但其数量不足以训练具备良好泛化性能的分类器的场景,我们提出了一个多源迁移学习框架。该框架主要包括两个步骤,第一步是在众多知识源中快速定位最佳的多个知识源;第二步是在每个入选的知识源上,使用风格迁移方法将目标域中的样本映射向知识源中对应类别内的原型,以消减目标域和知识源在条件分布上的差异。这样,知识源中的分类器就可直接用于推断目标域中样本的情绪标签。最终,在多分类器集成的框架下,我们实现了对目标域的高精度分类。该方法不仅可以利用目标域中的有标签样本,也可以纳入其中的无标签样本进行辅助学习。前者针对的是监督学习场景,后者针对的是半监督学习场景。因此,我们的方法具备良好的伸缩性和适应性。

3. 提出了一种基于隐含表示相似性的领域自适应方法

该方法面向无监督学习场景,针对的是目标域中完全没有标签信息的情况。该方法的基本思路是,一方面最小化知识源上的情绪分类误差,一方面在合适的距离测度下缩小两个领域的距离,以便在目标域上实现可靠的情绪标签推断。为了挖掘更多领域无关的数据结构,我们在神经网络产生的隐含表示上做知识迁移。在网络的浅层,我们使用对抗训练的策略迫使网络学习领域无关的特征表示,在此阶段,并不考虑情绪分类任务,仅要求两个领域数据的边缘分布相类似;在网络的深层,我们使用关联强化手段,以情绪分类任务中的标签一致性为约束,降低两个领域关于情绪标签的条件分布之间的差异。通过浅层和深层上不同策略的配合,我们分别在边缘分布和条件分布上降低了领域之间的差异,且在基准数据库上获得了截至目前最好的结果。

Other Abstract

Emotion Recognition facilitates a more friendly human-computer interaction, and provides a solid foundation in the area of security, as well as the diagnosis, treatment and rehabilitation of various mental diseases. Comparing with traditional emotion recognition methods such as facial expression, body movement and language, the Electroencephalogram (EEG)-based methods are more stable and reliable. As the difference of EEG distributions across subjects are huge, emotion recognition models are typically trained individually, which means we should collect substantial labeled data for each subject. It greatly limits the convenience of emotion recognition.

From the perspective of Machine Learning, this thesis attempts to confront the subject-to-subject variability in EEG, making the emotion classifiers applicable among different subjects. When the labeled samples for a new subject are scare (or even none), we combine information from previous subjects to obtain a reliable model for the new subject. According to the labeling situation of the new subject, we adapt three kinds of strategies, i.e., the supervised, the semi-supervised, and the unsupervised method. The main contributions of the thesis are summarized as follows:

1. A Novel Emotion Recognition Method Based on Deep Learning

Traditional methods concatenate EEG features from multiple channels as feature vectors, and then establish the mapping from the feature vectors to emotion labels with machine learning techniques. However, these methods do not consider the electrode placement directly. To incorporate the information of the electrode placement into the model training, we design a topology-preserving convolutional neural network (CNN). The shallow layers excavates the local responses of adjacent electrodes, while the deep layers are responsible for integrating the patterns of the whole brain. This method not only takes into account the response patterns of individual electrodes, but also considers the relationship between electrodes. The results show that the proposed method outperforms traditional methods, and achieves high-precision classification with low signal-to-noise ratio (SNR) EEG. On the basis of the above works, we transfer model parameters across subjects with finetune techniques.

2. A Multisource Transfer Learning Framework for Emotion Recognition

We regard the EEG of the new subject as target domain, and the EEG of previous subjects are source domains. For the scenario where there are a small number of labeled samples in the target domain, while their number is not sufficient to train classifiers with good generalization ability, we propose a multisource transfer learning framework. The framework mainly includes two stages. The first stage is source selection aimed at preventing negative transfer, and the second stage is to learn a style transfer mapping (STM) between the target and each selected source. STM reduces the domain differences in conditional distribution by mapping the target samples to emotion prototypes defined in the source domain. In this way, the source classifier could be used in the target domain. Finally, under the classifier ensemble scheme, we achieve high classification accuracy in the target domain. The proposed method could not only use the labeled samples in the target, but also include unlabeled samples to improve the performance. The former belongs to supervised learning, and the latter belongs to semi-supervised learning. Therefore, the method has good flexibility and adaptability.

3. A Novel Domain Adaptation Based on Latent Representation Similarity

This method is oriented towards an unsupervised learning scenario, where there are no labelled samples in the target domain. The basic idea of the proposed method is to minimize the emotion classification error on the source domain, while reducing the distances between the two domains, so as to achieve reliable emotion deduction results on the target domain. Instead of performing domain adaptation on the two domains directly, we do knowledge transfer on their representations generated by neural networks. The representations may capture more domain-invariant structures. In the first layers, we use adversarial training strategy to compel the network to learn domain-invariant representations, where we do not consider the emotion classification task, and the goal is to make the marginal distributions statistically similar. However in the last layers, we reduce the conditional distribution differences with association reinforcement based on round-trip label consistency. Through the coordination of different strategies in the first layers and the last layers, we reduce the domain differences in the marginal distributions and the conditional distributions, simultaneously. We compare our method with traditional transfer learning algorithms and recent network-based methods, finding the proposed method yields state-of-art performance.

Pages134
Language中文
Document Type学位论文
Identifierhttp://ir.ia.ac.cn/handle/173211/23899
Collection类脑智能研究中心_神经计算及脑机交互
Recommended Citation
GB/T 7714
李劲鹏. 脑电情绪识别中跨被试迁移学习方法研究[D]. 中国科学院自动化研究所. 中国科学院自动化研究所,2019.
Files in This Item:
File Name/Size DocType Version Access License
博士学位论文公开版.pdf(4425KB)学位论文 开放获取CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[李劲鹏]'s Articles
Baidu academic
Similar articles in Baidu academic
[李劲鹏]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[李劲鹏]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.