Knowledge Commons of Institute of Automation,CAS
ADFCNN: Attention-Based Dual-Scale Fusion Convolutional Neural Network for Motor Imagery Brain-Computer Interface | |
Tao, Wei1,2; Wang, Ze3,4; Wong, Chi Man1,2; Jia, Ziyu5![]() | |
发表期刊 | IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING
![]() |
ISSN | 1534-4320 |
2024 | |
卷号 | 32页码:154-165 |
通讯作者 | Wan, Feng(fwan@um.edu.mo) |
摘要 | Convolutional neural networks (CNNs) have been successfully applied to motor imagery (MI)-based brain-computer interface (BCI). Nevertheless, single-scale CNN fail to extract abundant information over a wide spectrum from EEG signals, while typical multi-scale CNNs cannot effectively fuse information from different scales with concatenation-based methods. To overcome these challenges, we propose a new scheme equipped with attention-based dual-scale fusion convolutional neural network (ADFCNN), which jointly extracts and fuses EEG spectral and spatial information at different scales. This scheme also provides novel insight through self-attention for effective information fusion from different scales. Specifically, temporal convolutions with two different kernel sizes identify EEG mu and beta rhythms, while spatial convolutions at two different scales generate global and detailed spatial information, respectively, and the self-attention mechanism performs feature fusion based on the internal similarity of the concatenated features extracted by the dual-scale CNN. The proposed scheme achieves the superior performance compared with state-of-the-art methods in subject-specific motor imagery recognition on BCI Competition IV dataset 2a, 2b and OpenBMI dataset, with the cross-session average classification accuracies of 79.39% and significant improvements of 9.14% on BCI-IV2a, 87.81% and 7.66% on BCI-IV2b, 65.26% and 7.2% on OpenBMI dataset, and the within-session average classification accuracies of 86.87% and significant improvements of 10.89% on BCI-IV2a, 87.26% and 8.07% on BCI-IV2b, 84.29% and 5.17% on OpenBMI dataset, respectively. What is more, ablation experiments are conducted to investigate the mechanism and demonstrate the effectiveness of the dual-scale joint temporal-spatial CNN and self-attention modules. Visualization is also used to reveal the learning process and feature distribution of the model. |
关键词 | Convolutional neural networks (CNNs) motor imagery (MI) brain-computer interface (BCI) self-attention mechanism |
DOI | 10.1109/TNSRE.2023.3342331 |
关键词[WOS] | EEG ; CLASSIFICATION ; TRANSFORMER |
收录类别 | SCI |
语种 | 英语 |
资助项目 | The Science and Technology Development Fund |
项目资助者 | The Science and Technology Development Fund |
WOS研究方向 | Engineering ; Rehabilitation |
WOS类目 | Engineering, Biomedical ; Rehabilitation |
WOS记录号 | WOS:001144547700020 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/55474 |
专题 | 脑图谱与类脑智能实验室 |
通讯作者 | Wan, Feng |
作者单位 | 1.Univ Macau, Fac Sci & Technol, Ctr Cognit & Brain Sci, Ctr Artificial Intelligence & Robot,Dept Elect & C, Taipa, Macau, Peoples R China 2.Univ Macau, Inst Collaborat Innovat, Taipa, Macau, Peoples R China 3.Macau Univ Sci & Technol, Fac Innovat Engn, Macao Ctr Math Sci, Taipa, Macau, Peoples R China 4.Macau Univ Sci & Technol, Fac Innovat Engn, Resp Dis AI Lab Epidem Intelligence & Med Big Data, Taipa, Macau, Peoples R China 5.Chinese Acad Sci, Inst Automat, Brainnetome Ctr, Beijing 100190, Peoples R China 6.Hefei Univ Technol, Dept Biomed Engn, Hefei 230009, Peoples R China 7.Univ Sci & Technol China, Dept Elect Engn & Informat Sci, Hefei 230027, Peoples R China 8.South China Univ Technol, Sch Comp Sci & Engn, Guangzhou 230027, Peoples R China |
推荐引用方式 GB/T 7714 | Tao, Wei,Wang, Ze,Wong, Chi Man,et al. ADFCNN: Attention-Based Dual-Scale Fusion Convolutional Neural Network for Motor Imagery Brain-Computer Interface[J]. IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING,2024,32:154-165. |
APA | Tao, Wei.,Wang, Ze.,Wong, Chi Man.,Jia, Ziyu.,Li, Chang.,...&Wan, Feng.(2024).ADFCNN: Attention-Based Dual-Scale Fusion Convolutional Neural Network for Motor Imagery Brain-Computer Interface.IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING,32,154-165. |
MLA | Tao, Wei,et al."ADFCNN: Attention-Based Dual-Scale Fusion Convolutional Neural Network for Motor Imagery Brain-Computer Interface".IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING 32(2024):154-165. |
条目包含的文件 | 条目无相关文件。 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论