CASIA OpenIR  > 多模态人工智能系统全国重点实验室
A multimodal approach to estimating vigilance in SSVEP-based BCI
Wang, Kangning1,2; Qiu, Shuang2,3; Wei, Wei2; Zhang, Yukun2,3; Wang, Shengpei2; He, Huiguang3; Xu, Minpeng1,4; Jung, Tzyy-Ping1,4,5; Ming, Dong1,4
Source PublicationEXPERT SYSTEMS WITH APPLICATIONS
ISSN0957-4174
2023-09-01
Volume225Pages:16
Corresponding AuthorQiu, Shuang(shuang.qiu@ia.ac.cn) ; Ming, Dong(richardming@tju.edu.cn)
AbstractBrain-computer interface (BCI) is a communication system that allows a direct connection between the human brain and external devices, which is able to provide assistance and improve the quality of life for people with disabilities. Vigilance is an important cognitive state and plays an important role in human-computer interac-tion. In BCI tasks, the low-vigilance state of the BCI user would lead to the performance degradation. Therefore, it is desirable to develop an efficient method to estimate the vigilance state of BCI users. In this study, we built a 4 -target BCI system based on steady-state visual evoked potential (SSVEP) for cursor control. Electroencephalo-gram (EEG) and electrooculogram (EOG) were recorded simultaneously from 18 subjects during a 90-min continuous cursor-control BCI task. We proposed a multimodal vigilance estimating network, named MVENet, to estimate the vigilance state of BCI users through the multimodal signals. In this architecture, a spatial -temporal convolution module with an attention mechanism was adopted to explore the temporal-spatial infor-mation of the EEG features, and a long short-term memory module was utilized to learn the temporal de-pendencies of EOG features. Moreover, a fusion mechanism was built to fuse the EEG representations and EOG representations effectively. Experimental results showed that the proposed network achieved a better perfor-mance than the compared methods. These results demonstrate the feasibility and effectiveness of our methods for estimating the vigilance state of BCI users.
KeywordVigilance estimation Brain -computer interface (BCI) Graph neural network Electroencephalogram (EEG) Steady-state visual evoked potential (SSVEP) Multimodal fusion
DOI10.1016/j.eswa.2023.120177
WOS KeywordEEG ; ATTENTION ; DECREMENT ; SYSTEM ; LEVEL
Indexed BySCI
Language英语
Funding ProjectBeijing Natural Science Foundation[7222311] ; Beijing Natural Science Foundation[J210010] ; National Natural Sci- ence Foundation of China[U21A20388] ; National Natural Sci- ence Foundation of China[62276262] ; National Natural Sci- ence Foundation of China[62206285] ; National Natural Sci- ence Foundation of China[62201569]
Funding OrganizationBeijing Natural Science Foundation ; National Natural Sci- ence Foundation of China
WOS Research AreaComputer Science ; Engineering ; Operations Research & Management Science
WOS SubjectComputer Science, Artificial Intelligence ; Engineering, Electrical & Electronic ; Operations Research & Management Science
WOS IDWOS:000989204800001
PublisherPERGAMON-ELSEVIER SCIENCE LTD
Citation statistics
Cited Times:3[WOS]   [WOS Record]     [Related Records in WOS]
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/53378
Collection多模态人工智能系统全国重点实验室
Corresponding AuthorQiu, Shuang; Ming, Dong
Affiliation1.Tianjin Univ, Acad Med Engn & Translat Med, Tianjin 300072, Peoples R China
2.Chinese Acad Sci, Inst Automat, Lab Brain Atlas & Brain Inspired Intelligence, State Key Lab Multimodal Artificial Intelligence S, Beijing 100190, Peoples R China
3.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China
4.Tianjin Univ, Coll Precis Instruments & Optoelect Engn, Tianjin 300072, Peoples R China
5.Univ Calif, Swartz Ctr Computat Neurosci, San Diego, CA 92093 USA
First Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Corresponding Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Wang, Kangning,Qiu, Shuang,Wei, Wei,et al. A multimodal approach to estimating vigilance in SSVEP-based BCI[J]. EXPERT SYSTEMS WITH APPLICATIONS,2023,225:16.
APA Wang, Kangning.,Qiu, Shuang.,Wei, Wei.,Zhang, Yukun.,Wang, Shengpei.,...&Ming, Dong.(2023).A multimodal approach to estimating vigilance in SSVEP-based BCI.EXPERT SYSTEMS WITH APPLICATIONS,225,16.
MLA Wang, Kangning,et al."A multimodal approach to estimating vigilance in SSVEP-based BCI".EXPERT SYSTEMS WITH APPLICATIONS 225(2023):16.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Wang, Kangning]'s Articles
[Qiu, Shuang]'s Articles
[Wei, Wei]'s Articles
Baidu academic
Similar articles in Baidu academic
[Wang, Kangning]'s Articles
[Qiu, Shuang]'s Articles
[Wei, Wei]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Wang, Kangning]'s Articles
[Qiu, Shuang]'s Articles
[Wei, Wei]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.