CASIA OpenIR  > 自然语言处理团队
Fine-grained neural decoding with distributed word representations
Shaonan Wang; Jiajun Zhang; Haiyan Wang; Nan Lin; Chengqing Zong
Source PublicationInformation Sciences

fMRI word decoding refers to decode what the human brain is thinking by interpreting functional Magnetic Resonance Imaging (fMRI) scans from people watching or listening to words, representing a sort of mind-reading technology. Existing works decoding words from imaging data have been largely limited to concrete nouns from a relatively small number of semantic categories. Moreover, such studies use different word-stimulus presentation paradigms and different computational models, lacking a comprehensive understanding of the influence of different factors on fMRI word decoding. In this paper, we present a large-scale evaluation of eight word embedding models and their combinations for decoding fine-grained fMRI data associated with three classes of words recorded from three stimulus-presentation paradigms. Specifically, we investigate the following research questions: (1) How does the brain-image decoder perform on different classes of words? (2) How does the brain-image decoder perform in different stimulus-presentation paradigms? (3) How well does each word embedding model allow us to decode neural activation patterns in the human brain? Furthermore, we analyze the most informative voxels associated with different classes of words, stimulus-presentation paradigms and word embedding models to explore their neural basis. The results have shown the following: (1) Different word classes can be decoded most effectively with different word embedding models. Concrete nouns and verbs are more easily distinguished than abstract nouns and verbs. (2) Among the three stimulus-presentation paradigms (picture, sentence and word clouds), the picture paradigm achieves the highest decoding accuracy, followed by the sentence paradigm. (3) Among the eight word embedding models, the model that encodes visual information obtains the best performance, followed by models that encode textual and contextual information. (4) Compared to concrete nouns, which activate mostly vision-related brain regions, abstract nouns activate broader brain regions such as the visual, language and default-mode networks. Moreover, both the picture paradigm and the model that encodes visual information have stronger associations with vision-related brain regions than other paradigms and word embedding models, respectively.

KeywordNeural Decoding Fmri Word Decoding Word Embedding Models
Indexed BySCI
Document Type期刊论文
Recommended Citation
GB/T 7714
Shaonan Wang,Jiajun Zhang,Haiyan Wang,et al. Fine-grained neural decoding with distributed word representations[J]. Information Sciences,2019(507):256-272.
APA Shaonan Wang,Jiajun Zhang,Haiyan Wang,Nan Lin,&Chengqing Zong.(2019).Fine-grained neural decoding with distributed word representations.Information Sciences(507),256-272.
MLA Shaonan Wang,et al."Fine-grained neural decoding with distributed word representations".Information Sciences .507(2019):256-272.
Files in This Item: Download All
File Name/Size DocType Version Access License
1-s2.0-S002002551930(2867KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Shaonan Wang]'s Articles
[Jiajun Zhang]'s Articles
[Haiyan Wang]'s Articles
Baidu academic
Similar articles in Baidu academic
[Shaonan Wang]'s Articles
[Jiajun Zhang]'s Articles
[Haiyan Wang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Shaonan Wang]'s Articles
[Jiajun Zhang]'s Articles
[Haiyan Wang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: 1-s2.0-S0020025519307820-main.pdf
Format: Adobe PDF
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.