CASIA OpenIR  > 类脑智能研究中心  > 神经计算与脑机交互
基于深度学习的单侧肢体不同关节运动想象解码研究
马学林
Subtype博士
Thesis Advisor何晖光
2020-11-26
Degree Grantor中国科学院大学
Place of Conferral中国科学院自动化研究所
Degree Discipline模式识别与智能系统
Keyword脑-机接口,运动想象,单侧肢体,认知失联,深度学习
Abstract

脑-机接口(Brain-computer interface,BCI)系统通过采集、分析大脑信号, 将其转换为输出指令,从而跨越外周神经系统,实现由大脑信号对外部设备的直 接控制,进而用于替代、修复、增强、补充或改善中枢神经系统的正常输出。基 于运动想象(Motor Imagery, MI)的 BCI 系统是唯一不需要外界刺激、能反映使 用者自主运动意识的 BCI 范式,其在运动功能代偿和运动功能修复方面具有重 要意义。近年来,MI-BCI 相关研究已经取得了一定的进展,但是仍然存在一些 瓶颈问题:1)运动想象任务解码性能有待提高;2)目前 MI-BCI 系统只能实现 基于左、右手等大肢体或单一关节的控制,并不能满足同一肢体不同关节的精细 控制需求。

本文面向 MI-BCI 研究中的难点问题,构建基于深度学习的高精度解码策 略,设计同一肢体不同关节的精细运动想象范式,探索其大脑响应模式,构建面 向 MI 任务结合 EEG 先验知识且具有可解释性的运动想象解码框架。本文研究 将推动精细运动想象 BCI 控制系统的研制,摆脱想象任务与外设输出模式不一 致的困境,为 MI-BCI 系统在康复领域的进一步应用提供关键技术基础。

  论文的主要工作和创新点归纳如下:

1. 针对 MI-BCI 系统不同肢体运动想象识别精度低的问题,本文首次提出 基于脑电电极空间位置的从额叶到枕叶空间序列数据的输入组织形式,并提出 了时间-空间并行长短期记忆网络框架来同步学习脑电信号中的时间-空间序列 信息。最后,时-空两条通路在特征层进行融合并分类。本文利用 eegmmidb 公开 数据集开展实验,实验结果证明了该方法的有效性,相比于经典运动想象解码方 法,本文所提出的时间-空间并行 LSTM 网络框架在不同肢体运动想象识别任务 上准确率提高了 8.25%。

2. 针对不同肢体运动想象范式对外控制时存在的认知失联问题,我们提出 了单侧肢体多个关节的精细运动想象新范式,并设计了通道-相关卷积网络用于 解码。我们设计实验并采集了 25 名被试的右侧上肢手指关节、手肘关节运动想 象任务的脑电数据以及睁眼静息数据。通过时频分析等发现两者都存在明显的 对侧占优效应,在对侧运动区导联的高 alpha 频率段存在显著差异。本文提出了基于通道-相关卷积网络的集成学习框架,从通道之间的相关系数特征中学习有 区分的表征,实现了 87.03% 的解码精度,相比于当前最好结果 74.2% 提升了约 13%,该项工作证明了单侧肢体两关节运动想象的可分性。

3. 针对单侧肢体运动想象指令集有限的问题,我们进一步拓展了单侧肢体 多个关节的运动想象范式,并提出频段注意力网络用于解码。我们设计实验和采 集了 20 名被试的五分类(指、腕、肘、肩和静息)数据集,分析了其大脑激活 模式。我们提出了频段注意力网络框架,该框架引入了注意力机制对多频带共空 间模式(common spatial filter, CSP)滤波特征进行特征权值再分配,并通过滑窗 保留了样本的时序信息,时序信息随后被 LSTM 层学习。在所采集的五分类数 据集上该网络获得了 46.15% 的解码精度,优于当前可比较方法。注意力模型权 重的可视化结果表明本文提出的网络通过学习能够关注到与任务相关的 alpha 和 beta 频段特征,且能根据输入自适应地调整注意力权重。该工作拓展了单侧肢体 运动想象范式的指令集,也为基于深度网络的脑电解码模型的可解释性研究提 供了新的思路。

Other Abstract

Brain computer interface (BCI) system collects and analyzes brain signals and con- verts them into output instructions, which can span the peripheral nervous system and realize the direct control of external devices by brain signals, which can be used to replace, repair, enhance, supplement or improve the normal output of central nervous system. Motor imagery (MI) - based BCI system is the only BCI paradigm that does not need external stimuli and can reflect the user’s autonomous motor consciousness. It is of great significance in motor function compensation and motor function repair. In recent years, the research on MI-BCI has made some progress, but there are still some bottleneck problems: 1) the decoding performance of motor imagery task needs to be improved; 2) at present, MI-BCI system can only realize the control based on left and right limbs or single joint, and can not meet the fine control requirements of different joints of the same limb.

Aiming at the difficult problems in MI-BCI research, this paper constructs a high- precision decoding strategy based on deep learning, designs fine motor imagery paradigm of different joints of the same limb, explores its brain response mode, and constructs an interpretable decoding framework for MI tasks combined with EEG prior knowledge. This study will promote the development of fine motor imagery BCI control system, get rid of the dilemma of inconsistency between imagination task and peripheral out- put mode, and provide key technical basis for further application of mi-bci system in rehabilitation field.

The main work and innovation of this paper are summarized as follows:

1. In order to solve the problem of low decoding accuracy of different limb motor imagery in MI-BCI system, the input organization form of spatial sequence data from frontal lobe to occipital lobe based on spatial position of EEG electrodes is proposed for the first time, and a temporal spatial parallel Long short-term memory network (LSTM) framework is proposed to synchronously learn the temporal and spatial sequence infor- mation of EEG signals. Finally, the temporal and spatial paths are fused and classified in the feature layer. In this paper, the EEGMMIDB public dataset is used to carry out experiments. The experimental results show the effectiveness of our proposed method. Compared with the classical decoding methods, the proposed temporal-spatial parallel LSTM network framework improves the accuracy rate by 8.25% in different limb motor imagery decoding tasks.

2. Inordertosolvetheproblemofcognitivedisconnectionindifferentlimbmotor imagery paradigms, we propose a novel fine motor imagery paradigm for multiple joints from the same limb, and design a channel correlation convolutional network for decod- ing. We designed the experiment and collected the EEG data of the right upper limb finger joint, elbow joint motor imagery task and eye opening resting data of 25 subjects. Through time-frequency analysis, it is found that both of them have obvious contralat- eral dominant effect, and there is significant difference in the high alpha frequency band of contralateral motor area lead. In this paper, an integrated learning framework based on channel correlation convolution network is proposed. The discriminative represen- tation is learned from the correlation coefficient features between channels, and the decoding accuracy is 87.03%, which is about 13% higher than the current best result of 74.2%. This work proves the separability of motor imagery of two joints from the same limb.

3. Tosolvetheproblemoflimitedcommandsetofunilaterallimbmotorimagery, we further extend the motor imagery paradigm of multiple joints from the same limb, and propose frequency attention network for decoding. We designed experiments and collected dataset of five categories (finger, wrist, elbow, shoulder and rest) of 20 sub- jects, and analyzed their brain activation patterns. We propose a frequency attention net- work framework. In this framework, attention mechanism is introduced to redistribute the feature weights of multi-band common spatial filter (CSP) features. The temporal information of samples is retained by sliding window, and then learned by LSTM layer. On the collected five classification datasets, the network achieves a decoding accuracy of 46.15%, which is better than the current comparable methods. The visualization results of attention model weight show that the proposed network can focus on the task-related alpha and beta band features through learning, and can adjust the attention weight adaptively according to the input. This work expands the instruction set of unilateral limb motor imagery paradigm, and provides a new idea for the interpretability research of EEG decoding model based on deep network.

Pages144
Language中文
Document Type学位论文
Identifierhttp://ir.ia.ac.cn/handle/173211/42204
Collection类脑智能研究中心_神经计算与脑机交互
Recommended Citation
GB/T 7714
马学林. 基于深度学习的单侧肢体不同关节运动想象解码研究[D]. 中国科学院自动化研究所. 中国科学院大学,2020.
Files in This Item:
File Name/Size DocType Version Access License
马学林毕业论文1207【最终版】.pdf(15176KB)学位论文 开放获取CC BY-NC-SAView
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[马学林]'s Articles
Baidu academic
Similar articles in Baidu academic
[马学林]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[马学林]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: 马学林毕业论文1207【最终版】.pdf
Format: Adobe PDF
This file does not support browsing at this time
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.