ADFCNN: Attention-Based Dual-Scale Fusion Convolutional Neural Network for Motor Imagery Brain-Computer Interface

doi:10.1109/TNSRE.2023.3342331

CASIA OpenIR > 脑图谱与类脑智能实验室

	ADFCNN: Attention-Based Dual-Scale Fusion Convolutional Neural Network for Motor Imagery Brain-Computer Interface
	Tao, Wei 1,2; Wang, Ze 3,4; Wong, Chi Man 1,2; Jia, Ziyu5 ; Li, Chang 6; Chen, Xun 7; Chen, C. L. Philip 8; Wan, Feng 1,2
发表期刊	IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING
ISSN	1534-4320
	2024
卷号	32 页码:154-165
通讯作者	Wan, Feng(fwan@um.edu.mo)
摘要	Convolutional neural networks (CNNs) have been successfully applied to motor imagery (MI)-based brain-computer interface (BCI). Nevertheless, single-scale CNN fail to extract abundant information over a wide spectrum from EEG signals, while typical multi-scale CNNs cannot effectively fuse information from different scales with concatenation-based methods. To overcome these challenges, we propose a new scheme equipped with attention-based dual-scale fusion convolutional neural network (ADFCNN), which jointly extracts and fuses EEG spectral and spatial information at different scales. This scheme also provides novel insight through self-attention for effective information fusion from different scales. Specifically, temporal convolutions with two different kernel sizes identify EEG mu and beta rhythms, while spatial convolutions at two different scales generate global and detailed spatial information, respectively, and the self-attention mechanism performs feature fusion based on the internal similarity of the concatenated features extracted by the dual-scale CNN. The proposed scheme achieves the superior performance compared with state-of-the-art methods in subject-specific motor imagery recognition on BCI Competition IV dataset 2a, 2b and OpenBMI dataset, with the cross-session average classification accuracies of 79.39% and significant improvements of 9.14% on BCI-IV2a, 87.81% and 7.66% on BCI-IV2b, 65.26% and 7.2% on OpenBMI dataset, and the within-session average classification accuracies of 86.87% and significant improvements of 10.89% on BCI-IV2a, 87.26% and 8.07% on BCI-IV2b, 84.29% and 5.17% on OpenBMI dataset, respectively. What is more, ablation experiments are conducted to investigate the mechanism and demonstrate the effectiveness of the dual-scale joint temporal-spatial CNN and self-attention modules. Visualization is also used to reveal the learning process and feature distribution of the model.
关键词	Convolutional neural networks (CNNs) motor imagery (MI) brain-computer interface (BCI) self-attention mechanism
DOI	10.1109/TNSRE.2023.3342331
关键词[WOS]	EEG ; CLASSIFICATION ; TRANSFORMER
收录类别	SCI
语种	英语
资助项目	The Science and Technology Development Fund
项目资助者	The Science and Technology Development Fund
WOS研究方向	Engineering ; Rehabilitation
WOS类目	Engineering, Biomedical ; Rehabilitation
WOS记录号	WOS:001144547700020
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：2[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/55474
专题	脑图谱与类脑智能实验室
通讯作者	Wan, Feng
作者单位	1.Univ Macau, Fac Sci & Technol, Ctr Cognit & Brain Sci, Ctr Artificial Intelligence & Robot,Dept Elect & C, Taipa, Macau, Peoples R China 2.Univ Macau, Inst Collaborat Innovat, Taipa, Macau, Peoples R China 3.Macau Univ Sci & Technol, Fac Innovat Engn, Macao Ctr Math Sci, Taipa, Macau, Peoples R China 4.Macau Univ Sci & Technol, Fac Innovat Engn, Resp Dis AI Lab Epidem Intelligence & Med Big Data, Taipa, Macau, Peoples R China 5.Chinese Acad Sci, Inst Automat, Brainnetome Ctr, Beijing 100190, Peoples R China 6.Hefei Univ Technol, Dept Biomed Engn, Hefei 230009, Peoples R China 7.Univ Sci & Technol China, Dept Elect Engn & Informat Sci, Hefei 230027, Peoples R China 8.South China Univ Technol, Sch Comp Sci & Engn, Guangzhou 230027, Peoples R China
推荐引用方式 GB/T 7714	Tao, Wei,Wang, Ze,Wong, Chi Man,et al. ADFCNN: Attention-Based Dual-Scale Fusion Convolutional Neural Network for Motor Imagery Brain-Computer Interface[J]. IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING,2024,32:154-165.
APA	Tao, Wei.,Wang, Ze.,Wong, Chi Man.,Jia, Ziyu.,Li, Chang.,...&Wan, Feng.(2024).ADFCNN: Attention-Based Dual-Scale Fusion Convolutional Neural Network for Motor Imagery Brain-Computer Interface.IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING,32,154-165.
MLA	Tao, Wei,et al."ADFCNN: Attention-Based Dual-Scale Fusion Convolutional Neural Network for Motor Imagery Brain-Computer Interface".IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING 32(2024):154-165.