毕业生知识库

基于运动想象(motor imagery，MI)的脑-机接口(brain-computer interface，BCI)系统解码人的运动意图，使人可以通过想象运动控制外部设备。MI-BCI系统不需要外界刺激、能反映使用者自主运动意识的BCI范式，可以替代、恢复、改善受损的神经系统功能，也可以用来控制机械臂、无人机、智能家居等。当前，MI-BCI系统存在识别正确率较低的问题，这极大地限制了MI-BCI系统的实际应用。为了解决这一问题，本文针对MI-BCI系统包括的机器和人两个主体开展解码和训练两方面研究，
论文主要工作和创新点具体如下:

1. 针对MI-BCI解码算法在目标被试训练数据量小时正确率低的问题，本文提出滤波器组动态权重领域自适应方法。用已有源域数据辅助目标域训练模型，提高目标被试样本少时识别正确率。本方法通过推土机距离衡量源域和目标域分布差异，通过对抗训练拉近源域、目标域分布；以分类别对抗的方式拉近源域、目标域条件分布。采用滤波器组方法从多个频带拉近源域、目标域分布。使用注意力网络动态的为各个频带分配权重，从而更好地融合多频带识别结果。实验结果表明本方法在各种训练数据量下均能提升识别正确率，在少一组训练数据情况下可以实现与已有方法相近的正确率。该研究能够减少MI-BCI校准时间，提高识别正确率，提升MI-BCI便捷性、实用性。

2. 针对MI-BCI在单一模态下解码正确率低的问题，本文首先设计实验并采集多模态运动想象数据集，然后提出时空对齐多模态运动想象解码模型。根据信号特点设计脑电和近红外模态特征提取网络，提出对齐两个模态空间特征的模态内和模态间表征优化损失函数，拉近两个模态空间表征。提出多模态融合模块，通过注意力网络对齐脑电和近红外时间特征并融合多模态特征。该研究首先采集了15名被试基于脑电-近红外的五分类MI数据集，从时、空、频角度对所采集的数据进行了分析。然后在自采集数据集和一个公开多模态运动想象数据集上进行了实验。结果表明该方法能够实现64.4%识别正确率，相较对比方法提高5.7%。可视化结果显示该方法能够提高多模态特征可分性。该研究通过多模态解码提升MI-BCI正确率，提高MI-BCI使用体验和实用性。

3. 针对MI-BCI被试运动想象能力不足、产生的脑响应不强导致识别正确率低的问题，本文提出基于分布适配运动想象反馈算法的训练系统，并对12名被试进行了在线运动想象反馈训练。该系统在人训练过程中更新反馈解码模型，并通过为样本加权重使模型适应人在反馈训练过程中脑信号模式的变化，使其在接下来的训练中为人提供更好的引导。该工作提出了基于样本分布的权重分配算法和基于共空间模式的有权重MI解码方法，搭建了基于视觉的在线运动想象反馈训练系统，设计在线运动想象反馈训练实验并采集12名被试在3种不同实验条件下的反馈训练数据。实验结果表明被试采用本方法训练相较对比方法能够提高训练速度，提高MI正确率。该研究提升被试运动想象能力，进而提高MI-BCI系统正确率和使用体验。

本文围绕MI-BCI系统，从人和机器两个角度出发，提出了基于领域自适应的运动想象解码方法，基于时空对齐的多模态运动想象解码方法，提升了MI-BCI的解码正确率；提出了基于分布适配反馈算法的运动想象训练系统，通过训练提升系统使用者的想象能力，从而提升系统正确率。这些工作为MI-BCI系统应用提供了有力的理论和技术支持，推动MI-BCI系统在医疗、非医疗领域的广泛应用。

Brain-computer interface (BCI) system based on motor imagery (MI) decodes human motor intentions, enabling individuals to control external devices by imagining movement. The MI-BCI system is the only BCI paradigm that does not require external stimulation and can reflect the user's autonomous movement consciousness. It can be used to replace, restore, and improve damaged neural system function, as well as to control mechanical arms, drones, smart homes applications, and other devices. Currently, the MI-BCI system has the problem of low decoding accuracy, which greatly limits its practical application. To address this issue, this dissertation conducts research on both decoding and training for machine and human subjects involved in the MI-BCI system respectively.
The main contributions of this dissertation are as follows:

1. To address the problem of low decoding accuracy of the MI-BCI decoding algorithm with small training data, a domain adaptation method is proposed. Existing source domain data is used to assist in training the model for the target domain, improving recognition accuracy when there is limited data for the target subject. This method measures the distribution differences between the source and target domains using the earth mover's distance and uses adversarial training to narrow the gap between the source and target domain distributions. Class-specific adversarial training is used to align the conditional distributions of source and target domain, while the filter bank method is used to narrow the distribution in multiple frequency bands. An attention network is used to dynamically allocate weights to each frequency band to better integrate the recognition results across multiple bands. Experimental results show that this method achieves better decoding accuracy compared to existing methods under each amount of training data on the public dataset, and achieves comparable accuracy with one less run of training data. This research reduces the data collection time required for MI-BCI use, enhancing its convenience and practicality.

2. To address the issue of low decoding accuracy of single modality MI-BCI, a multimodal decoding method based on temporal spatial feature alignment and fusion is proposed. EEG and fNIRS modality specific feature extraction networks are designed based on the signal characteristics. Intra-modality and inter-modality representation optimization loss functions are proposed to align the feature spaces of two modalities. A multimodal fusion module is proposed to align and fuse temporal features of EEG and fNIRS using attention network. The study first collected a five-class MI dataset based on EEG-fNIRS and analyzed the data from the temporal, spatial, and frequency domains. Then, experiments were conducted on a self-collected dataset and a publicly multimodal motor imagery dataset. Results show that the proposed method achieved 64.4% decoding accuracy, which is 5.7% higher than the best compared method. Visualization results show that the proposed method can improve multimodal feature separability. This study improves MI-BCI accuracy and enhances MI-BCI user experience and practicality through multimodal decoding.

3. To address the issue of low decoding accuracy of MI-BCI caused by insufficient motor imagination ability and weak brain responses of human subjects, a distribution-adaptive motor imagery feedback algorithm is proposed. This method updates the feedback decoding model during MI training and adapts the model to the changes in brain signal patterns of subjects during feedback training by weighting the samples, providing better guidance for the subjects in the following training. This study built an online motor imagery feedback training system based on visual feedback and proposed a weight allocation algorithm based on sample distribution and a weighted MI decoding method based on common spatial pattern algorithm. The study conducted online motor imagery feedback training experiments and collected feedback training data from multiple subjects under different experimental conditions. The experimental results show that the proposed method can improve the training speed and decoding accuracy of the subjects. This study improves the motor imagery ability of human subjects, making MI-BCI available to more people, and improves the accuracy and user experience.

This dissertation focuses on improving the accuracy of the MI-BCI system. From the perspectives of humans and machines, it proposes three part of work, including transfer learning using existing data to improve the accuracy, multimodal decoding using various brain signals to improve the accuracy, and distribution adaptation feedback training improves people's imagination ability. This dissertation provide powerful theoretical and technical support for the application of MI-BCI systems, promoting the widespread use of MI-BCI systems in both medical and non-medical fields.